Coding for the Big Picture

A Categorical Treatment of Romance

2011-01-11T23:44:00.001-08:00

p.p1 {margin: 0.0px 0.0px 13.0px 0.0px; font: 9.0px Arial} p.p2 {margin: 0.0px 0.0px 13.0px 0.0px; font: 13.0px Arial} span.s1 {color: #ffffff} span.s2 {text-decoration: underline}

Or, why Information Theory is more touchy-feely than classical E&M.

You folks are familiar with the concept of a duality, yes? Then what is the usual loser scenario for guys? You dote on some girl with all this attention but then she runs away with some motorcycle dude, right? And to add insult to injury, she complains about him to you afterwards.

What would be duality of this situation? Just reverse the arrows, yes? I don't want you to get to an answer without thinking about it, so take some time to muse a little, then highlight the text after this sentence. You're a girl. No one calls you, not even that motorcycle dude you gave it up for. "I hope he dies in an accident. Asshole." ;-)

Now there is a catch here, because this seems to have violated conservation of attention. It's like integrating the magnetic flux over a closed continuous surface, yknow? Should sum to zero.* Alas, the physically correct description of the world is totally useless to us. We have a glimmer of hope, however. See, if you step back and think about it,... If you give unconditional attention, is it still attention? ;-) Remember, it takes only one bit to encode unconditional attention.** One. Bit. :-) Admittedly, this reconciliation requires Information Theory, but what matters is that the math works!! :-D ***

I wonder how many analogies I can butcher with dualities, and which supplementary theories I will need to grease the axe uh I mean logic...

[*] Maxwell's equation for magnetic flux works particularly well here when you imagine how the positive and negative nature of the interaction would play out. ;-) And the fact that it sums to zero. Face it, kid. There is no magnetic monopole.

[**] Kindly consider that girls may find one-bit guys to be cute, but ultimately a little annoying. And that giving it up to those cool boys riding motorcycles can also be encoded in one bit. Happy romancing, and try to not say anything stupid! :-)

[***] The solution to maths is… MOAR MATHS!

Posted via email from lambdasquirrel's posterous

An import trick too useful to pass up

2010-11-08T15:51:00.001-08:00

So here is a trick I learned from reading the Snap code over the last week.

Namespace collisions suck. Data.Map, Data.Set and Data.List all have fairly similar functions that we all know and love to use, and they differ subtly, so people often import them qualified, i.e.

import qualified Data.Map as M

import qualified Data.Set as S

import qualified Data.List as L

Now the annoying thing about this is that then you have to prepend the type signatures too, e.g.

foobar :: S.Set a -> a -> S.Set a

This is pissy, so what some genius who worked on Snap did was:

import Data.Map (Map)

import qualified Data.Map as M

import Data.Either (Left,Right)

Now this sounds simple and all, but it actually works Much Better in practice than in theory, partly because type constructors like Left and Right rarely overlap from module to module.

Posted via email from lambdasquirrel's posterous

How do you tell good code from bad anyway?

2010-11-08T15:40:00.001-08:00

So here's one straightforward metric for code cleanliness. Is there code that could've been purely functional that can't be easily be extracted from the monadic wrappers you placed it in? I guess a similar thing for OCaml would've applied to the OO and functional code.

This is probably not as easy as it seems though, because there's plenty of stuff that would've looked like it could've been pure but is actually much better off effectful. Sometimes there's no substitute of experience.

Posted via email from lambdasquirrel's posterous

Untitled

2010-09-10T22:13:00.001-07:00

A few months ago, there was a useful tutorial on using CouchDB with Haskell. You can find the original here.

One weakness of these DB layers is that you have to verify your data, and the APIs to the input data are usually not typesafe. As it turns out, it's incredibly easy to use type-level programming to make your DB calls typesafe.

Lets take an idealized version of a typical DB get call:

getDBUnsafe :: (JSON a) => DB -> String -> IO a getDBUnsafe = undefined

There's two weak spots here. The first is the part where we pass in the DB key, and the second is when we use the value. The latter is more insidious than the former, because whatever it is you're using to parse your JSON, it's probably a pure function, so if you're coding in the usual expedient way, you'll have no idea why the parse is failing, when the real cause is that you're fetching from the wrong DB.

So here's the framework for a solution that uses FunctionalDependencies and MultiParamTypeClasses to impose a constraint on the type of the DB key and stored value, based on the type of the DB.

class (JSON v) => DBTy a k v | a -> k, a -> v where getDBName :: a -> String getKey :: a -> k -> String getDB :: (DBTy a k v) => a -> k -> IO v getDB db k = getDBUnsafe (getDBName db) (getKey db k)

So how we use this? Well you just define a dummy type for a DB like such:

type UserId = Int data Avatar = Avatar ByteString deriving (Eq, Show, Ord, Typeable, Data) instance JSON Avatar where showJSON = toJSON readJSON = fromJSON data AvatarsDB = AvatarsDB String instance DBTy AvatarsDB Int Avatar where getDBName (AvatarsDB name) = name getKey _ = show

After that, you just replace your unsafe DB calls that look like this:

v <- getDBUnsafe "avatars" userId

with this:

v <- getDB AvatarsBB userId

Oh and here's the stuff you need to paste at the top of all this to get it to compile.

{-# LANGUAGE FunctionalDependencies, MultiParamTypeClasses, DeriveDataTypeable #-}
module FundepsExample where

import Data.ByteString.Lazy

import Text.JSON
import Text.JSON.Generic

type DB = String

I know it's a pretty silly example and use case, but I was surprised that there were folks in my local Haskell meetup who hadn't seen it, so I thought I should share it. It's saved me no end of errors ever since I put it to use.

Now in practice, I've found it more useful to create a KeyStringTy class instead of using that getKey bit. As it so happens, the sort of stuff I typically use to index my bit bucket database are also the sort of stuff that I use in RPC calls from the web. It's probably less correct, but it sure was dandier to code. Watch it bite me in the ass someday.

Posted via email from lambdasquirrel's posterous

I just had to post this ...

2010-08-17T15:52:00.001-07:00

Posted via email from lambdasquirrel's posterous

Mike Rowe Celebrates Dirty Jobs

2010-06-21T13:11:00.001-07:00

I couldn't agree with this guy more. The cultural dynamic has led to more than a few awkward conversations about how programming doesn't interest you anymore. Then there's some equally silly remark on how one could remedy the problem with -insert-deux-ex-machina- here. Or an even more awkward defensive remark about how X is still interesting, the other guy just hasn't seen aspect A of X.

It's nearly impossible to talk about things like work ethic and the psychology of motivation with such a dynamic. People tend to fall into one of three camps: that they do work simply because it has to be done (i.e. it's a responsibility, so shut up), that they do their work because of the money it earns them, or that they'll only do those things that pulls their heartstrings. You don't get very far when a conversation usually devolves into some sort of argument over philosophies or personality traits.

Perhaps that's why Self-Determination Theory works so well: it bypasses the problem entirely. In the context of a conversation however, I wonder whether the smart thing to do is to keep quiet because there does not seem to be a right answer. I personally came from the third camp, but doing a startup quickly teaches you that passion alone can't carry you the whole way.

I wonder how many other creative industries suffer from this problem, and in what way, because I'm sure that the problem manifests itself differently depending on the field.

Posted via email from lambdasquirrel's posterous

Haskell & STM, Why no Applicative?

2010-05-19T00:27:00.001-07:00

There was a fellow on #haskell the other day who was apprehensive about learning STM. Should he learn it after learning category theory? We assured him that Haskell's STM was fairly simple, whereas category theory is a dense liberal art that you study to enrich your mind. Someone pointed him to the wiki and he went on his way.

Afterwards, I perused the wiki (again). What struck me was that there was no example there that show you how to convert a plain old bunch of IO routines into STM routines. When I went to make one myself, I realized how brain-dead easy it is, but that there's something missing...

Anyway, here's a chalked up example where someone has to perform multiple time-consuming tasks (preferably in parallel), with the interesting routine in bold:

import Control.Applicative

import Control.Monad

import Control.Concurrent

import Control.Concurrent.STM

import Control.Concurrent.STM.TMVar

import Data.DateTime

main :: IO ()

main =

do putStrLn "Without STM"

t1s <- getCurrentTime

stuff <- withoutSTM

t1e <- getCurrentTime

putStrLn $ show stuff

putStrLn $ "That took " ++ show (diffSeconds t1e t1s) ++ " seconds"

-- this is the routine that actually does stuff

withoutSTM :: IO GroceryStore

withoutSTM =

do a <- getTomatoesCountFromDB

b <- haveFreshBerries

c <- getNameOfCurrentStore

return $ GroceryStore a b c

getTomatoesCountFromDB :: IO Int

getTomatoesCountFromDB =

do milliSleep 1000 -- simulate slow DB read

return 5

haveFreshBerries :: IO Bool

haveFreshBerries =

do milliSleep 1000

return True

getNameOfCurrentStore :: IO String

getNameOfCurrentStore =

do milliSleep 1000

return "Tom's Produce"

data GroceryStore

= GroceryStore

{ numTomatoes :: Int

, freshBerriesInStock :: Bool

, nameOfStore :: String

}

deriving (Eq, Show, Ord)

-- helpers

milliSleep = threadDelay . (*) 1000

So there you have the plain old imperative code.

And here is the same code modified to use STM.

-- getTomatoesCountFromDB, haveFreshBerries, getNameOfCurrentStore are unchanged from before

-- this is the modified routine

withSTM :: IO GroceryStore

withSTM =

do a <- stmFork getTomatoesCountFromDB

b <- stmFork haveFreshBerries

c <- stmFork getNameOfCurrentStore

GroceryStore <$> (stmWait a)

<*> (stmWait b)

<*> (stmWait c)

main :: IO ()

main =

do putStrLn "With STM"

t2s <- getCurrentTime

stuffWithSTM <- withSTM

t2e <- getCurrentTime

putStrLn $ show stuffWithSTM

putStrLn $ "That took " ++ show (diffSeconds t2e t2s) ++ " seconds"

return ()

-- more helpers

stmWait = atomically

stmFork :: IO a -> IO (STM a)

stmFork m =

do tmv <- newEmptyTMVarIO

let m' = m >>= (atomically . putTMVar tmv)

forkIO m'

return $ readTMVar tmv

Now the folks who've been playing with this for a while won't find this particularly remarkable, but I wouldn't have used it a year ago when I was first learning Haskell, so I thought someone should make note of it. There are also two important things to take away from this.

1. You could only do this if imperative routines are treated as values. The pons asinorum that Eric S. Raymond was referring to is not without its benefits.

2. Would an applicative instance for STM be nicer? At the least, we might be able to do code like this instead.

stmWait $ GroceryStore <$> a <*> b <*> c

What would be wrong with the naive implementation of this? e.g.:

(<*>) :: (Functor m, Monad m) => m (a -> b) -> m a -> m b

(<*>) mf ma =

do a <- ma

f <- mf

return $ f a

Again, I'm probably way out of my depth here, so I apologize profusely if this is asinine.

Posted via email from lambdasquirrel's posterous

Academia isn't Broken. We Are.

2010-05-05T07:51:00.000-07:00

I saw this piece on Hacker News this morning. It struck a chord, but something wasn't right about it.

http://brucejacob.tumblr.com/post/373498114/academia-and-the-decline-of-wealth-in-america

If academia is contributing to the lack of innovation in this country, then maybe it's because we expect the wrong things from academia? I don't mean to say this as another pompous American, but when I used to chat with friends from abroad back in school, I was struck by how many of them had a uniform educational experience. This wasn't a blanket effect and there were more than enough exceptions to produce many of the most awesome researchers I've met. Back at home though, even the countries that had effective programs to retain their top talent suffered from a lack of innovation.

By contrast, there is no standard curriculum at the top 5 US universities for CS. But most of the kids coming out there are shills anyway. I went to one of the good schools, and many kids (and their parents) were concerned about whether what they were learning would "prepare them for the real world". That basically meant: did it teach you Java or consulting? You see what's happening? What our education system didn't do to them, their own expectations of college did, and sadly, they seem to have done it to themselves just as badly as the education system of those foreign countries did.

But education isn't about churning out stamped spoons, and that's why crap like that No Child Left Behind Act bothered me so much. Where we went wrong is that we began viewing education as something everyone is entitled to, for all the wrong reasons. Education is not factory farming. Steve Jobs took calligraphy because he thought it was interesting, and no one thought what the Google guys were doing was "practical". I'll wager that the Google guys did it because it seemed like nerdy awesomeness to transform web search into a giant linear algebra problem. Just as Academia is about exploring the boundaries of what we know, Education is about enriching one's thought process, and that's the leading source of innovation in our modern economy: good ideas from the fringe, implemented intelligently and autonomously.

The entitlement of success that seems to follow from attending college is what's broken. The expectation that you will get a cushy 9-5 job in return for that diploma is what's broken. It in essence is a laziness of the mind, an unwillingness to chart out one's own path, the very idea of which is quite unacademic.

Dirty Tricks

2010-04-28T21:41:00.000-07:00

Here is a type I'm using to record stats.

data UserzFormationViews

= UserzFormationViews

{ formationsViewed :: [(FormationId, UTCTime)]

, ...

}

Basically, the data is a monoid. Actually, it's even less restrictive than that, because the monoidal append on this piece of data is commutative (i.e. embarrassingly parallel). It's incredibly convenient to be able to just throw a pile of these together and tabulate the stats with a familiar mconcat.

At other times, it's useful to get all the user's views, in a form like this:

class RetrieveViews a where

getViews :: a -> [(UserId,FormationId,UTCTime)]

This obviously messes with the monoidal definition of UserzFormationViews. The initial object (i.e. mempty) would need a dummy value for the UserId. In more primitive languages, you'd just leave that as NULL, which is a dirty hack. And we don't like those kinds of dirty hacks in Haskell. What do we do instead?

instance RetrieveViews (UserId, UserzFormationViews) where

...

vs = getViews (zeUserzId,zeUserzViews)

So wrong and so awesome at the same time. :)

A Problem I've Encountered with Types

2010-04-21T01:32:00.000-07:00

Everyone is familiar with how we can use types to encode checks into our code. For example:

data User
  = SystemUser SystemProcessInfo
  | AnonUser TimeLoggedIn
  | RegisteredUser UserId

data UserId
  = PaidUser (..)
  | FreeUser (..)

More generally, you can use type witnesses to mix together similar types in the same list, to mimic some aspects of dynamic typing.

The Problem:

Suppose you have a type witness, such that all the witnessed types derive some typeclass (or a similar situation where you are using types to enforce behavior). Is it possible to have the compiler automatically derive that typeclass for the type witness, i.e. something similar to GeneralizedNewtypeDeriving? This kind of boilerplate is yucky:

data A
  = A1 X
  | A2 Y
  | A3 Z

instance Foo A where
  getFoo (A1 x) = getFoo x
  getFoo (A2 x) = getFoo x
  getFoo (A3 x) = getFoo x

One area where this problem seems unavoidable is when you have to deal with collated data of different types. For example, say you have a list of items that were found to be complementary in terms of content, but are of different types. In such cases, type witnesses are unavoidable. If I can get a good solution to this problem, I'll be sure to post it.

Lazy IO with Haskell Monads

2010-04-09T01:41:00.000-07:00

Haskell's treatment of imperative programming and side effects is possibly the most derided feature of the language. As someone who's written desktop apps, I'm certainly aware of the inflexibility that Haskell's sandboxing imposes. However, monads happen to mesh very nicely with Haskell's purity and laziness, and what they take away, they give back in other ways.

Suppose for example, you are designing an evil robot that goes out and steals wheels from people's cars. The robot can steal up to 4 wheels from a car, but because it tries to avoid being detected by witnesses (organic or otherwise), it may abort the job prematurely. You need it to fetch 20 wheels a night. No more, no less. Fewer, and you may not be able to pay the bills. More, and the police might start catching on.

For this simple example, you have a function that returns a monadic value:

stealWheelz :: Coordinates -> IO [Wheel]

Then you have some other pre-calculated list of random locations that you want the robot to steal from.

locations :: [Coordinates]

Now here's the funky part. You can define a function takeM, as such:

takeM :: (Monad m) => Int -> [m [a]] -> m [a]

takeM = takeM' []

takeM' :: (Monad m) => [a] -> Int -> [m [a]] -> m [a]

takeM' acc 0 xs = return acc

takeM' acc n [] = return acc

takeM' acc n (x:xs) =

do rs <- x

let rs' = take n rs

n' = n - length rs'

takeM' (acc ++ rs') n' xs

and then your list of tires stolen can simply be:

takeM 20 (map stealWheelz locations)

With this code, the robot will only steal as many tires is as necessary, even if the list of coordinates is of infinite length. Because Haskell is lazy, and IO routines (and all imperative programming that are modelled as monads) are treated as lazy values, you can pass around tasks, with all their arguments bound--even in a strict monad like IO--, and not have to deal with a lot of the usual glue that goes into determining how many results to return.

Btw, that means you can actually pass around the value:

let stealSomeWheelz = map stealWheelz locations

and take results from that list of actions only as you need them. This sort of thing is doable, but *much* less elegant in languages that do not treat actions as if they were values. When you design a language to operate via side effects, then you have to deal with those side effects immediately, as you arise. That means you can't easily reason about it at a higher level, even when you're trying to do simple things like "take the first n results".

As an example, I can apply a filter function to those results, as such:

-- Police commissioner is in town tonight with his Escalade

let stealSomeWheelz = map stealWheelz locations

filter noCaddyWheelzTonight <$> stealSomeWheelz

I discovered this trick when I needed to sanitize the first n queries for a particular area of interest, given a statically-ranked listed of sources. In a more general setting, removing the glue from imperative programming means that I can focus more on what I'm doing, rather than what I'm programming. That's a good thing.

I'd love to see more gems like this. I'm sure someone else has discovered this already, it's in a library somewhere, and I'm just yakking the obvious. I have a feeling that the functional community (and the rest of the world by proxy) is only beginning to discover the value of treating imperative routines as if they were monadic values. Indeed, at the time of this writing, there's a video on HN concerning monads in Clojure.

Reviving Post-Modern Journalism

2010-03-31T02:07:00.000-07:00

Or, why newspapers aren't making money anymore.

1997 was a different time. I remember seeing all the adults on the subway with their newspapers in hand. Back then, in New York, the newspaper you read said something about you. The regular Joes read the Daily News or the Post, depending on their political slant. These were entertaining tabloids that slightly enhanced the truth, and you could tell just by reading the headlines. The folks who considered themselves somewhat more educated would have in hand a copy of the New York Times or the Wall Street Journal. It was harder to detect bias in those papers, but you could certainly discern it if you knew the raw facts and the spectrum of viewpoints.

Just as importantly, you could tell immediately which newspaper was which, just by looking at them. In the case of the plebeian pubs, the headlines were typed with screaming 2-inch high font. The patrician pubs used starkly different and recognizable typefaces for their titles.

Today, most newspapers in America are going out of business, and one thing that's perhaps been overlooked was the status conferred by reading a newspaper. One of the oldest adages in marketing states that the magnitude of status or identity conferred by a product determines how many brands you'll have of it. There are two major brands of toothpaste, as far as I can tell. People don't take too much pride in their toothpaste. You can't tell if someone brushed with Crest or Colgate, only whether they did at all. On the other hand, there's dozens of different car brands. Cars are expensive. They signal immediately one's level of wealth and some brands alert others to the owner's hobbies, or the size of the owner's beer gut or phallic insecurities.

No one really read the news just for the sake of knowing.

So how does this relate to newspapers?… It's quite simple: What if they don't confer status or identity anymore?

There's more than one take on this. My middle school math team teacher, a fairly well educated and openly intellectual person, once lamented how there were more newspapers and viewpoints in his day. He was born just before America suburbanized in the 50s, 60s and 70s. Sprawl was the first thing that happened. People may only see the newspaper on your lawn for a second in the morning, if at all. Next came computers, phones and tablets.

The problem with the modern world is that it's just not possible for others to see what you are reading anymore. What are they going to do? Squint at your laptop to see what you're reading? Forget about trying to surreptitiously read off an iPhone. You could argue that it can be done, but what's important is that people don't expect those around them to do so. Most damningly, most news in a paper is from the AP or Reuters anyway. It's just a piece of text and it really doesn't matter then where you get it from.

The other thing that happened is that the newspaper has stopped being a marker for appearing intelligent and well-informed. Old folks read newspapers. The well-informed folks get their news off the wire, on their smartphone, 30 seconds after Reuters or the AP pushed it out. What's post-modern journalism to do in the face of this?

Journalism needs to be "interesting" again, and other silly proposals.

I think that whatever journalism survives this new age will have to branch out again. It needs to once again appeal to people's inclinations and identity. Perhaps that's why blogs are so popular these days. They provide "analysis" alongside content. There are some news sources that seem to understand this. Stratfor, for example, provides a well-reasoned realpolitik attitude towards world news, and combines it with a solid understanding of local history and politics. It should be emphasized that unless you're aiming to be the 800lb gorilla like Fox News, your core readers should be willing to pay for your news. Everyone's talked about how the Internet has allowed companies to exploit niche markets. Perhaps it actually requires it, in our new world of commoditized information. Specialize. That's only part of the puzzle though.

The next part of the puzzle may be difficult for old media to stomach. They need to let subscribers share their material. If buying a newspaper was a way to confer status, then they can revive the dynamic in the modern world by letting subscribers mail a limited number of pieces to an unlimited number of their friends. Chances are, these friends aren't going to read the piece at all. What matters though is that subscriber will pay for a news service that tells others about how they think the world should be seen.

Sharing would also be a way for a subscription service to spread virally. People should also be able to post links to these articles on some social network, where they would be able to give props for them. People get a buzz out of seeing that others agree with their viewpoint, or found an interest in something that they read off the wire.

I actually think there's room on the web for a news service tailored exactly for this purpose (or maybe even more), especially if such a service were to aggregate based on people's interests and viewpoints. Implementing such a thing would of course be no trivial matter, but seeing how the incumbents seem to have dropped the ball, the time seems right. It could be monetized as well. The service could form a symbiosis with the news media by helping them sell subscriptions.

Études en OCaml: Keeping your functions clean, Pt 2

2008-07-04T22:07:00.000-07:00

Long, non-atomic functions are the bane of functional languages

For reasons I don't completely understand (but will try to discuss anyway), functional languages seem to raise the bar for code quality. In the interest of making life more exciting, functional languages also pile on the rewards when you do it right, and they'll conversely pie you in the face when you get it wrong.

You can accomplish a lot in a few lines of OCaml code, especially if what you're doing is well-suited to the functional model, and this creates special challenges for folks coming from C/C++. The tuple and variant matching features of the language make it fun and easy to violate separation of layers. C programmers will especially be delighted that their functions look so clean and short (only 20 lines!), and yet accomplish so much!

Alas, the actual usage of the function will inevitably be ugly, and you won't find that out until much later, when you have to refactor that 20 line function down to one or two wrapper functions and two or three 5-10 line functions. The problem with long functions is that it's difficult to reuse them, especially with the bread and butter fold_left and map functions. I hate to use buzzwords but I think it's accurate to say that your functions need to be *agile*.

The very idea of this starkly contrasts with how we write code in C/C++. Those primitive stone age tools limit how short one can practically write a function; the lack of built-in support for tuples and abstract datatypes means that the programming overhead of passing return values (via pointer arguments and structs) is fairly high.

As such, abusing the language is normal and expected for OCaml beginners, especially those coming from the imperative domain, so don't fret if you find yourself rewriting functions so that they're shorter. It is happening because you are learning.

Tools for writing clean functions

You can't truly fix a problem until you know why it happens, yes? If it's true that our predisposition for the detail-oriented is what keeps us from writing clean functions, then perhaps the fix is one of the oldest in the book: Think before you hack. If you're not scoffing, then here is one idea for abetting this. If you're not exactly sure what you're doing, write a comment-- one phrase --to describe what you want to do before you start writing your code. If you need another phrase, then you need another function.

I've collected a number of thoughts-- more concrete ones --that have also helped me when I was learning OCaml, and I hope you find them useful as well.

I have a hunch that clean code is not unlike a fractal...

OCaml-specific rules

- The broad 25-line rule

I pulled 25 out of my ass actually, but that's not really the point. The point of this exercise is to give beginners an idea of what OCaml code should look like. Readable OCaml code is typically composed of 5-20 line functions, or larger functions that are composed of subfunctions that are 3-13 lines long (assuming of course that you have spaces in between and not all 13 lines are actually code). Dispatches will be longer, but should not contain any logic of their own.

There will be many exceptions to this rule, particularly with purely imperative code like numerical computations, gui dispatchers and sysadmin-like tasks. OCaml has attracted a lot of numerical folks to its side and those jerks often like to remind me that none of my rules for keeping my code clean apply to them. It's important to identify purely imperative tasks like numerics, because they typically result in long functions no matter what you do. Once you have them identified, you can confine them into their own functions so that they won't muddle the rest of your code.

- Follow the types that you declare as layers of separation

The same H-M variant types that grant OCaml its incredible expressiveness are also useful guidelines for determining how you should write your functions. Fortunately for us, it's much easier to think in terms of what you want to hold in a type than it is to figure out where a function should begin and where it should end. In a nutshell, your functions may directly match against a particular type, but they shouldn't descend into the members of those types.

Good:

type entity =
metadata *
geometric_primitive

...

let render_geo_prim gp =
match gp with
TRIANGLE (a,b,c) -> ...

let render_entity (md,gp) =
render_geo_prim gp

Bad:

type entity =
metadata *
geometric_primitive

...

let render_entity (md,gp) =
match gp with
TRIANGLE (a,b,c) -> ...

I've tried applying this methodology at the module level. In a general sense, one could try to avoid descending into types that aren't declared in that module. I'm not sure if this works. Consider the following example. You have a graphics program. In this graphics program you have a type definition for the entities you want to draw, e.g. triangles, spheres, lines, meshes, etc. For the renderer you want the program to support a GL mode and a realtime-raytraced mode. Putting all of the code into the same file with the type definition would be a gross violation of layers, and will likely result in a single source file that'll be well over 100kb in size.

There are some types that you will want to expose the internals of. I term these types interface-types. Then there are some types that you expect to grow in the future, in ways that will totally mess with the pattern matching when you change the type spec. You typically want to hide the representations of the latter types, or perhaps consider using records instead of tuples. You can do this by writing a custom .mli header file for your source file, and then erasing the right hand side of the type definition. An easy way to do this is to pipe the output of `ocamlc -i`:

ocamlc -i blah.ml > blah.mli

blah.ml:
type stuff = wakka * float * int * string

blah.mli:
type stuff (* hide actual definition *)

Stuff that applies to OCaml and programming in general

-Resist any impulse to inline your code

I've heard many experienced and smart programmers tell me that I should inline a function because it's only got one caller. DO NOT DO THIS. Unless you are writing numerical computations or GL rendering routines, you *will* want the shorter function to List.map or Something.iter at some later point.

The jury is undecided on this in the realm of general programming. Although there are many good programmers who've told me that it's silly to split functions up, the best C/C++ programmers I know will typically do so. However, I care little for arguing how things are done with ancient fossils like C, so I leave the general applicability of this rule up to the reader.

- No more than 4 arguments for any given function

Why four? Cognitive psychologists have known for some time that people have a short term memory buffer of size 7, give or take 2. I'm sure many programmers will have the "max" of 9 (more if they've got Asperger's or autism which is perfectly possible), but you don't want your code to be only readable to the freaks (like yourselves) with photographic memory. Pushing the limits of your coworkers' memory isn't nice. Kindly leave them a free register.

"But I can't do with less than 5 independent arguments...", you say! If you *need* more than 4 independent, unrelated arguments for a function, it's possible that you need to write whatever it is you are doing in OO. If you need more than 4 arguments and they are not independent/unrelated, then perhaps you should be passing those values around as a tuple or as a record. I've found it helpful to try to understand why the OCaml designers implemented records, when there already tuples in the language. Functors are also sometimes useful for making code readable, and for keeping the argument count down.

It's not a bad idea to apply this analysis elsewhere. In OCaml, 1 line of code will often express 1 routine, as opposed to C, which often requires 10-20 lines to do something like traverse a list of any sort (even using library functions). Thus it's practical to have many functions that are only 3-7 lines long in OCaml.

Note that you'll have to amend this rule for older languages like C, where we commonly pass around 2-3 arguments that we barely need, because the language does not support tuples or OOP.

- No getters for the OO code

People with C++ and Java experience will revulse in horror at the idea that you can't make a class variable public, or even protected.

Having written OO code in several languages for some time now, I'm convinced that C++ and Java need to expose their class implementations and member variables because they lack functional expressiveness (e.g. tuples, abstract primitive datatypes, first class functions). In fact, one of the greatest epiphanies I've had came after I read that you generally should not have any getters at all. Try to do everything locally, without exposing any variables.

Ironically, I read the no-getters rule in a Python article somewhere, which itself sourced the tactic to an unnamed Java article! The idea applies to OCaml all the same. Not using any getters lives up to the spirit of what the OCaml OO restrictions are trying to accomplish: proper separation of layers in your OO code.

There will be corner-cases where the no-getters rule doesn't apply, but it's surprising how far you can get with this practice, and how much it improves your code. It also seems to be possible to shove all the variables that need getters into a separate class, usually called something like "SettingsForBlah".

Études en Ocaml: Keeping your functions clean, Pt 1

2008-06-30T14:08:00.000-07:00

"But you could save a PUSH instruction if you inlined it..."

One of my coworkers, who is still somewhat fresh out of college, asked me why I had broken up one of my functions into a few smaller ones, when there was only one caller for each of them. One of the funnier things he suggested was that inlining the code would save an x86 push instruction. But then he surprised me, by suggesting that unnecessarily breaking up functions can make code more difficult to read.

Why is it so hard to keep functions short?

It's hard to keep functions short. If you do not think so, then you are probably lucky, or smart, or perhaps it isn't too difficult for you after years of hacking code, but that doesn't make it any less inherently difficult. It is still useful to understand why other people find it difficult to keep functions short. Indeed, why is it difficult at all?

Before we continue, I should point out that it's better to say that functions should be "atomic", or "clean". A good function is one that can't logically be broken down anymore without making it pointless. This small but important semantic difference has significance later on.

What do you see here?

I'm going to make a grand, sweeping assertion that many of you will likely say "duh" to. If you disagree, then it's probably because it doesn't apply to you, and that's great but we're trying to figure out why keeping functions clean and short is difficult for most programmers-- including myself --, so please bear with me.

Programmers are by and large detail-oriented. At the least, we're good at details, because we have to be. Computers are finicky contraptions that break unless everything is exactly correct. There is no "almost works" in much of what we do. Because of their digital and binary nature, computers and the tools we use to program them are not fault tolerant. It is left to the programmer to make sure everything is right, and because of this, anyone who does not enjoy picking apart the details of a problem probably won't enjoy programming.

So how did I come to this conclusion? It's quite simple. Back in school, we had two pretty darn tough "weed-out" courses. One course was taught in a high level language, and the other course was taught in C and assembly. Sounds like *your* university doesn't it? Now let me ask you real quick, did the majority of your CS friends find the C/Assembly class easier, or did they totally pwn that Scheme/Lisp/SML/Haskell class? Which one did they like more? Most importantly, which one was regarded as "more practical"?

Our undergraduate advisors noted that people generally found the C/Assembly class easier. Most CS students also liked that class more. The ones who didn't usually went on to grad school, which serves to further skew the pool of industry programmers in the direction of the detail-oriented.

Programmers in industry who maintain others' code are subjected to an additional set of biases. It takes some time to learn to read code, and reading code is a bottom-up process, especially if the said code is written poorly (now isn't it funny the way that *everyone* works somewhere where the code sucks?). If you're new to the field (like my coworker) and you're still plying the code line-by-line, it wouldn't seem that there'd be any point to breaking functions up at all. What's more is that sometimes people break up functions without adding any atomicity to the structure of their code.

Since programmers are detail-oriented people, writing atomic functions is not something that comes naturally because we do not typically encounter our problems from the top down. This is especially true if we learned to program in a low level, iterative language like C. Writing clean, atomic functions requires you to understand consciously what it is you are trying to do, at a high level.

A tree or a forest?

But many programmers don't consciously know what they're trying to do. They just hack towards a solution and somehow, magically, a working piece of code appears, both to their benefit and to their detriment.

The awesome and frightening conclusion that we gather from this is that it is hard for many of us to write clean code simply because we *can* hack code. Our ability to meticulously design algorithms, to read and hack other people's code (often at a line by line basis), to find needles in haystacks (debugging, anyone?), and perhaps our own personalities --these things are what made us into programmers in the first place. Details matter: above all, the code must work! "We can worry about how messy it is after we ship. ;)"

But these traits predispose us to a mindset where we do not see our problems in the big picture sense that is useful for writing clean/atomic functions, writing and using black boxes, and maintaining a separation of layers. Indeed, the fact that we're told that we should write "short" functions is indicative of this mindset. Shortness has little semantic meaning. It is a detail, a direct metric, not an overarching goal like "cleanliness".

Coding for the Big Picture

A Categorical Treatment of Romance

An import trick too useful to pass up

How *do* you tell good code from bad anyway?

Untitled

I just had to post this ...

Mike Rowe Celebrates Dirty Jobs

Haskell & STM, Why no Applicative?

Academia isn't Broken. We Are.

Dirty Tricks

A Problem I've Encountered with Types

The Problem:

Lazy IO with Haskell Monads

Reviving Post-Modern Journalism

Études en OCaml: Keeping your functions clean, Pt 2

Études en Ocaml: Keeping your functions clean, Pt 1

How do you tell good code from bad anyway?