pl patterns

Meta-Problems

The other day, I overheard one co-worker showing Emacs to another co-worker. The one who was new to Emacs said something like, “Oh, it doesn’t do X?” The Emacs proponent replied, “Well, no, but you could always implement it if you want it.”

And this statement sounded really familiar. It sounded just like Paul Graham’s description of the Blub language (which is an amazing essay, I must say).

Now, I’m not saying some other editor is more powerful than Emacs. But I am saying that the fact that this statement was made is an indicator that Emacs isn’t the most powerful. It’s a Blub editor compared to the imagined ideal editor.

The entire idea of a Blub language and the power continuum doesn’t apply to just languages; it applies to all tools which are powerful enough to emulate tools with more power. With programming languages, as long as a language is Turing-complete, it can emulate any other language that is Turing-complete. A similar “whatever you can do with your tool, I can do with Blub” argument can be used with editors when Blub can be extended with plugins or scripts.

Emacs may be powerful due to its extensibility, but I think we can do better.

Patterns of Patterns

Building on top of the most powerful tools available, which eliminates usage patterns and implementation details, leads to new usage patterns. Eventually, these patterns will be considered implementation details. When the Internet was first invented, HTTP GET wasn’t an implementation detail; it was what they were creating — the churning fringe of development. Now it’s a detail, as we’re creating AJAX web apps built on top of HTTP and a slew of 57 other things that didn’t exist before.

Whatever you make, there will always be patterns — patterns of repetitive tasks that are tedious, expensive, and could be done cheaper by a machine. Thus arises the desire to automate the task completely, or at the very least, create a tool to make the repetitive parts disappear. When this is done, the pattern becomes embodied in the tool, and at first, all patterns seem to have been eliminated. But in time, new patterns emerge, in which your tool is just a small piece. At this point, you’re back where you started, possibly with more resources and personal growth if you’re lucky. But the loop begins again.

Seeing this meta-pattern, we can attempt to follow it to its conclusion. If we’re going to create tool X in the first tool-making iteration and tool Y in the second tool-making iteration, can we go directly to Y and skip X altogether? If you can see patterns 2 or more iterations ahead, you may still have to implement X but you’ll have a great advantage by seeing the longer-term goal of Y. That said, it’s hard to see patterns, even in this iteration, let alone iterations far in the future. Who knew, that in just 10 years, the web would evolve the way it did. Back in the 90’s, it wasn’t even clear that anyone could make money off of a search engine (one of the most lucrative fields) because the thinking was that people would search for something once, bookmark it, and never return to the search engine — a legitimate concern back in the day. Obviously wrong though knowing what we know now. …So can we do better?

If you noticed, what I just did was unroll the tool-making loop 2 times. We tried to exploit the pattern by jumping ahead 2 loop iterations at once. But to fully exploit a pattern, you must look at the big picture. You have to look at the entire loop all at once.

Most people are stuck in 1 — the current iteration. Last paragraph, we jumped to 2. What about n? The “limit”, if you will.

The first thing that sticks in my mind is the loop condition. When do we ever exit? Why are we here in the first place? There are many directions this line of questioning can take. One of them — if you pursue it faithfully to its conclusion by looking for the limit of the {meta to the power n}-problem — leads to questions like “What do I want out of life?”, “What is my purpose?”, and “What is my true essence that defines me?”. But those are beyond the scope of this essay.

Another direction we can take is to observe the fact that as long as we are in this loop, we are struggling to find a pattern, factor out that pattern, and make the thing we originally set out to make (at the non-meta level). And this is all irrespective to what we’re actually making. In other words, as long as you’re making new things and competing for resources, regardless of the field, whether it’s software, clothing, or underwater basket weaving, you will always be in this loop.

To What End?

When I was younger, I thought solving certain problems on the computer was valuable. I began programming to solve some of them.

But after a while, I observed the act of creating software as being repetitive. The act of programming — the very act of problem-solving — became the barrier to solving problems. The problem-solving embodied in programming became the new problem. It was a meta-problem because I didn’t actually care about writing the programs themselves; I cared about the solutions that the programs entailed. Writing program after program, it only made sense to solve the meta-problem. And that is how I got interested in programming languages, the medium through which all programs were communicated.

Looking back, what I did was jump from 1 to 2. But for a while, I failed to make the quantum leap from 2 to n.

The human race will no doubt continue to improve technology ad infinitum, becoming more efficient and allowing people to do things they could never do before. And for short-term goals, this makes sense. But how is that any better overall?

I used to think that advancing technology to do work for us was a good thing. It would free us up to focus on what really mattered.1 But why not just focus on what really matters right now?

Some people will say that they can’t because of some thing X that prevents them. To that, I would argue that you will have to learn how to deal with X, because even if you solved X (which humans will one day), there will always be Y to fill the role of the thorn in your side. Of course, some will say that advancing the state of the human race in the way I describe is what matters. And to that, I would say you’ve never honestly asked yourself “why”.

When you’re being practical in the short-term, the thing you’re actually making on the non-meta level is what matters. But when you’re setting long-term goals, the solution to the meta-problem is what matters, because the problem (just like in software) inevitably arises again in an infinite number of variations. It only makes sense to come up with a general solution.

But you can apply the same reasoning to the meta-problem itself, and think of it as simply another problem. Now you care about the solution to the meta-meta-problem, and it makes no sense to settle for a solution to merely the meta-problem. …Ad infinitum.

What does all this “meta” talk mean? It means that technology is inane — it can’t solve your Ultimate Problem. There will always be another problem. So should we give up and stop advancing technology?

It only makes sense to do something that has no inherent value if it works towards value elsewhere. So, if for example, you were using the day-to-day work of building the next great web app that allowed people to eat ice cream in a way that was socially networked (with a news feed of who ate what flavor combinations when (and the number of pounds they gained as a result (updated in real-time (and notified via text message)))), then go right ahead. Ditto for creating AI or reciting 1024 digits of π while juggling if you believe it is a means to something inherently valuable.

But to think for one moment that it in itself will satisfy you in the long-run, that it is a valuable end in itself, you’re deluding yourself.

---

Stand on the Shoulders of Giants

I feel like I haven’t even written a useful piece of code in ages because the mere thought of boilerplate code stops me in my tracks. This is one of the main motivations behind creating a new language free of hindering boilerplate code. But then you start running into other problems.

The egotistic developer will believe that creating a new programming language is the key, secretly striving for the silver bullet, even though he would never openly admit it because he is not even conscious that he is doing it. Some people believe that the silver bullet is Lisp, but for some strange reason, it has failed to slay the dragon for the past 50 years and counting.

The more seasoned developer, unambitious and static, will believe that creating a new language is a waste of time due to how many wheels have to be re-invented in a new language before it even approaches the usefulness of existing languages. Not to mention the fact that any feature could simply have been implemented in (or on top of) the existing language in the first place.

I think the only thing that makes sense is to strike a balance with something that allows (an order of magnitude) more succinctness and extensibility without sacrificing the millions upon millions of man-hours already spent by armies of programmers. We can stand on the shoulders of giants like IBM, Microsoft, Sun, and Google, and move forward without taking a giant leap back.1

On that note, we have people building things like Instapaper. And Instapaper is great; I use it. But it’s silly that I now use 2 completely separate bookmarking services that are oblivious to each other.

Of course, upon closer inspection and further use, I realized that the way I use del.icio.us and the way I use Instapaper are completely orthogonal. That is, the set of links that I save to del.icio.us and the set of links I save to Instapaper are completely disjoint. Occasionally I will first save a link to Instapaper, read the article, and then save it to del.icio.us, but that is a rare exception.

However, it’s completely obvious to me that the functionality of Instapaper is a proper subset of the functionality of del.icio.us. In other words, everything that Instapaper can do, del.icio.us can do and more.

Why then didn’t Marco build Instapaper over del.icio.us? It seems like a perfect match.

With the Web 2.0 craze, APIs started popping up everywhere. Even tools like Pipes and Popfly to wire those services together. But who is using them?

Instapaper’s value is solely in its amazingly simple interface. The reason I use it differently from del.icio.us is because its interface affords to different things. It makes different operations cheap, and that changes the way I think about those operations. But why doesn’t Instapaper integrate with my del.icio.us account, simply tagging things with a “read-later” tag? When it comes to bookmarking, del.icio.us is king; there’s no disputing that. Instapaper wouldn’t have less value if it were built on top of del.icio.us, it would actually have more.

And this, in my humble opinion, is the next step for the web. People will finally start realizing that most of the functionality in their little web app to-be is already done — not in a library — but in a web service.2

Services that do single things — and do them right — will be indispensable to the web. And a glue language will be in high demand. But personally, as an entrepreneur, I’m not looking towards Yahoo Pipes or Microsoft Popfly (yes, in direct opposition to the idea of standing on the shoulders of giants). I want to own my mashups, and those services don’t give me that at all.3

---

1, 2, n

It’s the 1, 2, n rule in software development.

You do something once. You do something twice. And the next time you do it, you generalize it for all times henceforth.

Waiting longer to generalize is painful at best. In the worst case, it leads to maintenance nightmares as when the same thing is done 14 times in 14 different places, each time slightly different than the rest.

Generalizing too soon however leads to other problems. Everyone’s done this at one point in their coding career or another. Hey, this is a great idea! Why hasn’t someone else done it before? I’ll code it completely general the first time around so that it’ll be done right, from the start. And then a few weeks of coding go by, you start to actually use it, and you find out the hard way that either it was a really bad idea or the API is so terrible that you have to re-write it. Because basically, you didn’t iterate your design.

Code is all about design. And good design is re-design.

This is a big part of the reason why programming languages are so far behind. The amount of work that it currently takes to create (a compiler for) a programming language is so high, that the time it takes for a language designer to get feedback on his design is years. But by that time (if he was lucky enough to actually get users), he’s already invested so much into it that he is unlikely to throw it away and start from scratch — something a good designer must do. On top of that, even if he wanted to start from scratch, it would take many months to de-program his mind from thinking in the previous language. (Like the first time a Java programmer learns Haskell and tries to write a loop!)

An IDE is part of a language. Error messages are part of a language. Interacting with other systems, written in other languages, is part of a language. They’re all part of the interface of the language. And the interface is everything.

It’s a chicken-vs.-egg problem though. Because to get all these tools made, based on history, requires that a language be popular. But how do you get people to use a language if it doesn’t have the most basic tools like a debugger?

JavaScript is doing it right now. JavaScript tools — which make using the language no longer so painful — are cropping up everywhere these days. And the way it did that was by being the only choice if developers want their code to be executed in a browser window. Most languages aren’t so lucky. So what is a language designer to do?

As a designer, I can’t simply make a command-line compiler for my language. I have a vision of what editing code is like. And it’s not with a text-editor. The closest thing I’ve ever seen to a true code-editor is Eclipse for editing Java code, but Eclipse has been around long enough for us to learn a few lessons. I can already hear the Emacs junkies crying about how Emacs is the ultimate editor. But again, it’s still a glorified text-editor.

A true code-editor must absolutely be aware of the semantics of the language you are editing. It is inseparable from the compiler. Because the compiler knows how to interpret your code, but when something is amiss, it must be able to feed back to the programmer.

Writing code is all about design. And a designer needs feedback — as much feedback as possible. The quicker he gets that feedback the better, because he can make any necessary changes to the design only after he’s gotten feedback on his current design.

So any code-editor that doesn’t give immediate feedback on parse errors and requires that you do a full build is simply crap! How can you possibly work like that?! Ditto for type errors, link errors, and test-assertion errors. People see it as obvious when the old-timers tell stories about how they used to submit compilation jobs to mainframes on punch-cards, only to find out the next day that they forgot a semicolon and had to re-submit the entire job. Yet when it comes to where we are, here and now, people are blind.

Frankly, I’m getting tired of it. I want to see test data flow through my code as I write it and the unit-test results change from red to green as I fix a bug. I want to be able to highlight two blocks of code and have my code-editor factor out a function, replacing the two blocks with applications. I want bottlenecks to be highlighted in-editor based on profiling results. And compilation should be done completely in the background so I don’t have to wait after making a change to use it. And this read-eval-print-loop that’s all the rage nowadays might actually be useful if I could use it in the context of a particular point of my code.

Is it really that hard? Someone please tell me I’m a visionary because I can’t believe that this can’t be done. Nothing a little universal substitution and computing power can’t solve.

Methods to the Aha

When I was a fresh CS major at CMU, I took the standard 15-251 Great Theoretical Ideas of Computer Science class. The professor Steven Rudich introduced us to some amazing topics. Even though I was never able to use it to its full potential, the most valuable thing taught was what he called the “aha! method”. In other words, he taught that there was a method to getting to the point where a light bulb goes off in your head and you get an insight that allows you to solve a hard problem, making you say “aha!”. And moreover, that this method can be learned.

The list he gave was as follows.

• Phrase Hygiene
• Representation
• Induction
• Modularity
• Exemplification
• Refinement
• Abstraction
• Bracketing

Briefly, he meant: tag assertions with phrases that indicate your degree of conviction; actively choose your representation of the problem; use the various forms of induction; break the problem into smaller subproblems; use small examples; don’t give up after you’ve found an answer; abstract away the non-essential details of the problem; try to give upper and lower bounds and make them converge. I’m not going to go in to more details of each of them, but the overall idea is that each is a trick to help you more easily see the pattern in the problem that will lead you to a solution.1

As Rudich pointed out, in any field — martial arts, violin, tennis, magic, programming — the novice makes a huge motion, the black belt makes a small motion, and the master makes a tiny motion. The more complete the pattern you see, the less work is required to accomplish a task.

Recently, I’ve found a new method. What made me see this was an “interview question” I heard way back when I was in Rudich’s class. It goes something like this… You have to cross a gorge that’s 50 feet wide and 50 feet deep. All you have is two 20-foot ladders and all the rope you need. How do you do it?

And no, you can’t go around it. What now? (You might want to actually think about this before reading on.)

Select text here for the spoiler… The answer is — since you have all the rope you need — that you fill the gorge with rope, and walk across.

Now, the answer to these riddles is often obvious in hindsight. But what can we learn from it? Why couldn’t we see the answer before? It wasn’t until recently that I realized that if you abstract away the details of this solution, what you had to do was substitute an abundance of one resource for the lack of another resource. And this I’ve discovered   is another method to the “aha!”: Universal Substitution.

What I mean is that any resource can be substituted for any other resource. And this is simply amazing!

…As I mentioned in my last post, analogy seems to be a pattern of all of these. The Representation method above seems to hint at that. But how do you know which representation to choose? You have to already have made a connection — an analogy — with another problem.

Jeff Hawkins seems to think that all problems are solved through either memory or analogy. In other words, you either remember the solution to the problem if you’ve seen it before. Or you relate the problem to another problem and remember the solution to that, and then apply that solution to the specific situation at hand   which is something your mind does all the time.2

If this is true (which seems to make sense), then analogy is a problem-solving method. But not only that, it is a method to finding problem-solving methods! It is a method of solving any problem, even meta-problems and meta-meta-problems.3

So this leads us to our next question. How do you become good at creating analogies?

Analogies are connections between two different things. They are the result of abstracting something away from two things that are different and being left with things that are the same. Analogies are patterns.

One thing I’ve noticed that increases the likelihood of seeing patterns is trying different things — all sorts of things. Oftentimes you suddenly realize that things you’ve become so used to don’t have to be the way they are. That it is not necessarily the right way or the wrong way. But also, when it’s different enough, you can’t help but see what isn’t different. In other words, what remains constant — which is a pattern.

You never know where a pattern might show up, or what lines it might cross.4 Plus, oftentimes the analogies within a field have already been exhausted by others, while cross-discipline analogies lay un-reaped.

Also, there is the trap of becoming too close to something. After working on a problem arduously for a while, it is often helpful to first take a break, and then step back and look at the problem to get a bigger perspective. In other words, disentangle yourself from the details and take a look a the whole picture at once, just as if you were putting together a jigsaw puzzle.

All this really is, is another example of the law of Balance.

I’ve actually attempted to devise an algorithm for finding patterns before. The thing is, there are a seemingly infinite number of ways to categorize things. So if you try to categorize things and then look for something re-occurring, you’ll never know if you’re simply using the wrong categorization. That’s why it’s hard to learn something new in general. According to Jeff Hawkins, you have to see a pattern before you can learn anything. The people who tend to be seen as smart are the ones who pick up new things quickly, and they can do this because they see the patterns quicker than others. But it is a completely unconscious activity to them.

This begs the question though, is there another way of finding patterns? Can we find patterns without first categorizing things?

It’s like if you have a hash function that abstracts away from things you’re looking at, and you keep hashing different things hoping to find a collision. Once you do, you’ve found a pattern. For example, if your hash function took physical objects as input and gave a color as output, you could hash all the things on your desk and if any items resulted in the same color, you would have found a commonality between those two things, which is a simple yet significant pattern.

But what if you’re looking for another pattern? Which hash function should you use? Is there another way besides this hashing method?


See also: Side-Stepping Obstacles in Space and Time

---