pl patterns

Blub by Convention (In Defense of Arc)

The entire Arc language is just a few Scheme macros. …Or so the criticism says.

I can’t use Common Lisp. mapcar, lambda, destructuring-bind, etc. These are things I use all the time, yet they have some of the longest, most unnecessary names.

Scheme is the Java of Lisps. define, begin, list-ref, etc. It was seemingly designed to be as descriptive as possible, and in doing so, also became as verbose as possible.

Yes, it’s true that I can redefine all those things. But at that point I’ve created Arc. So if you see the pain of using mapcar and list-ref, then in a nutshell, you see the need for Arc. Period.

In the same way that Perl has functional programming features, but the community and conventions don’t use it that way, which means the libraries aren’t designed for functional code, hindering your use of it, so too are Common Lisp and Scheme hindering. The conventions hinder the use of its full potential.

Lisp is potential. Untapped potential. Arc is an attempt at tapping it.

The blub paradox implies that once you truly get the features of Lisp — first-class continuations, condition-signals and restarts, macros, serialization of code to syntactically valid code — all the imitation features like function objects, exceptions, Ruby’s catch/throw, the C-preprocessor, and serialization to XML are… disappointing at best. Atrociously unusable at worst. This is why Lisp is more powerful than non-Lisps.

But Arc is better than other Lisps because Lisp by definition is flexible. And flexibility is a double-edged sword. You can define anything to be anything — both in a good way and a bad way. Arc was specifically designed to pull out common patterns and be as concise as possible. If it’s true that Arc is just a bunch of macros, then these macros that I would have had to define myself in Common Lisp or Scheme are part of the core language or conventions of Arc. So instead of only me knowing about those macros, all Arc programmers know about them.

Code that does not follow a convention is more difficult to integrate with code that does, so much so, that the fact that libraries exist using a convention is a huge incentive to use the same convention. Additionally, if I write code that does not follow the convention, people who maintain my code inevitably re-write it in their own image. It happens slowly over time perhaps, but inevitably. Every one of those names (mapcar, lambda, list-ref), and dozens more in the core language and standard libraries, is a kludge. But if I redefined them, the first thing someone else touching my code would do is look at it, say “wtf!”, and delete my redefinitions and change everything back to the canonical use. In Arc, on the other hand, instead of my code degrading to the conventions of Common Lisp over time as I move on and others take over my code, it holds the high caliber of the conventions of Arc.

However, if you actually succeed at changing all those kludge names in Common Lisp, you will have improved the language by an order of magnitude. And when you improve something by an order of magnitude, you haven’t made something better, you’ve made something new. (Hence, Arc is genuinely new.)

Basically, non-Lisps are blub compared to Lisp because they limit what you can abstract. However, once you have a dialect of Lisp, there are so few visible barriers to abstraction. Instead, non-Arc Lisps are blub compared to Arc because the conventions adopted by the community — and lack of a ability to change those conventions — create barriers to flexibility, removing one of the very features that makes Lisp so great.

Put another way, non-Arc Lisps are blub by convention. The macros that make Arc Arc are a big deal because they are an attempt to change the convention.

What I am getting at is:

1. Many languages could be better than they are if they used different conventions.
2. It’s damn-near impossible to go against the conventions.
3. Conventions are difficult to change, more of a way of thinking of the community.
4. It’s worthwhile to push good conventions into the core language/libraries, and if you do that, you’re essentially making something new.

This has little to do with Arc, except that Arc is an example where someone (Paul Graham) tried to do #4 because of 1 through 3.

Alternative to XML Validation

The desire to validate the structure of a document has come up a number of times recently. Say you have a public service where one of its operations takes in a complicated tree-structured document. You want a way to:

1. use a programming-language-agnostic data format for your document;
2. formally specify the expected structure of the doc;
3. validate that input to calls to your service actually follow the structure;
4. and hopefully, allow clients to do this validation themselves, w/o hitting your service.

A well-known language-agnostic way of doing all this is with XML as your document format, an XML schema as your specification of the expected structure, and XML validation as the publicly known algorithm for validating.

Having done some type-theory, I would much rather say that my data format is a data structure in a programming language, the spec of the structure is a type declaration, and validation is type-checking.

So I’ve wondered, is there a solution to my problem 1 through 4 — without using XML?

A few well-known programming-language-agnostic data-formats are: s-expressions, JSON, YAML. I have found Kwalify for YAML, but not much else.

This is a shame because people who believe they might ever need 2 through 4, choose XML for 1 (pl-agnostic data format) simply because they don’t know of any alternatives. This sucks because if all you are doing is 1, then you’ve made the worst choice for the job.

XML sucks to read as a human b/c there is so much extra noise. It gets to the point where the majority of your effort spent reading XML is filtering. It also sucks to use in programming b/c the API is terrible. XML doesn’t map perfectly to the data-structures in most languages, even Java, its biggest fan.

So again, if all you’re using it for is 1 — a programming-language-agnostic data format — XML is one of the worst choices. Yet, the fact that XML dominates in 2 through 4, it remains the default.

Sending an IM via Gtalk in Clojure

Email has its place, but when it comes to short notifications — of the status of a job, for example — IMs work great.

If you think of your software as your co-worker — someone you work with who helps you do your job — all kinds of functionality just make sense. Would you rather your co-worker emailed you or IMed you? Then the software should do that.

This code snippet sends an IM using Smack 3.1.0, a Jabber client library (Apache 2.0 license).
Download the Smack jar.
Browse Smack source.

Note: you must invite the username to chat and confirm in Gmail before messages will get sent.

user> (def conf (org.jivesoftware.smack.ConnectionConfiguration. "talk.google.com" 5222 "gmail.com"))
#'user/conf
user> (def con (org.jivesoftware.smack.XMPPConnection. conf))
#'user/con
user> (.connect con)
nil
user> (org.jivesoftware.smack.SASLAuthentication/supportSASLMechanism "PLAIN" 0)
nil
user> (.login con "my.username@gmail.com" "mypassword" "resource")
nil
user> (.isAuthenticated con)  ;; check that we are authenticated
true
user> (def chat (.createChat (.getChatManager con) "send.to.username@gmail.com" nil))  ;; this nil is the listener -- put something here to receive IMs!
#'user/chat
user> ;; Invite to chat in Gmail -- sending messages won't work w/o doing this!
nil
user> (.sendMessage chat "it works!")
nil
user> (.disconnect con)
nil

Update: I created a project that uses this along with log4j, letting you pull a logger out of nowhere and send out status IMs.