Saturday, July 28, 2012

Software visionaires and opportunities

When I saw Steve Jobs in some of his presentations I thought: oh boy! You got it. It's all about simplicity and how to get things done the right way. An enabler for the masses. An opportunity for creating opportunities. Jobs was an enabler for co creation. He always defined Apple as a software company. And so it is, I think!

The exact same emotion came to my mind as I saw the amazing Rich Hickey keynote Simplicity Matters from the railsconf 2012. Fast forwarding the next five years I see more and more developers using functional programming, because the paradigms of software development seem to shift into a more data flow centric direction. At least, this is my very personal opinion. Internet, SQL, Linq, Reactive Programming, Parallel Computing. It's all about data and stream processing. And speaking of streams its the same exact thing as a list. Eventually, in the very near future functional programming will be used everywhere.

How do I achieve this task of data flow processing? The Internet is really the best example of how to do it. Data flow and functions that transform state. Not objects being responsible for tracking state. There are really good reasons to do it this way. And some others, most likely saving resources, to do it the OO style. I'm practicing the data flow approach even in python with its dict and list built in types. Actually, everything in python itself is an object containing dict's or list's. Dict and list are the most optimized data types in python and you'll see them everywhere floating around. Guido, AFAIK, is not a FP believer, so your mostly stuck in mutable OO paradigms. You're free to use whatever approach you like, but python will never be a real functional language. Never the less it's quite an awesome experience getting up to speed with python. There is data on the one side and functions on the other. Nowadays I dislike heavy class hierarchies for certain kind of problems.

Now what are the functional alternatives? You can walk along the traditional fashioned path the Lisp, Haskell, OCAML way of life or have a look at more modern approaches, entering the mainstream ready languages like F#, Scala or Clojure. Why? Because they run on top of enterprise grade VMs using the, and inter operating with libraries build over one or two decades. That's not an indicator for success, but it is a strategy for getting the masses to think about and use functional programming. It's a smooth way of migrating your skills and start new projects. I can easily play around with the concepts and integrate them step by step. This is simplicity from another point of view. As a vendor how can I attract the million developers out there to get in touch with? It's simplicity of getting used to the new tech. Take their comfort zone (.NET or Java) build a new language and let them adopt the new paradigm step by step. Don Syme has done it many years ago with F# and I think Rich Hickey with Clojure will be another great guy on the JVM side. Both are functional programming languages but differ in their approach.

Having used python as a real dynamic language the last 3 years in production I'm very comfortable with this modular style of programming. It's a pleasure to have all the freedom of choice and not being nagged by intriguing compiler error messages. No limitations and no inherent compiler laws. With great power comes great responsibility. And I decided to take this challenge 3 years ago. Now Clojure as a Lisp derivation is a real dynamic language, too. I don't know whether it is a good choice for enterprise grade solutions. I never really had a chance to test it. Twitter was built with ruby, and later replaced by java. Well, it's quite a different beast to deal with. A gazillion messages floating through the pipe. Python is simple to learn, really simple. Clojure is different, if you've never really been in touch with the Lisp style of programming - yet simple. Have a look at Clojure.core! You're free to invent whatever new "keyword" or operator you want. Just do it. hygienic macros and the way Lisp has been built. Amazing.

Simplicity is very subjective. Simple for me can be hard to manage for anyone else and vice versa. I'm very pleased with the amount of code I have to write to solve a problem. Give it a try and get inspired by simplicity.


  1. How are you going to manage complexity in real-life FP systems? Lots of functions, composed in an tangled mess? I know only one approach (used by Erlang) - the concept of 'processes' sending messages to each other. Interestingly, those processes effectively are objects in true OO sense. So, I think any non-trivial FP program inevitably ends up to be composed of some kind of objects. So, what's the point in not using an OO language from the beginning?

    1. Composition of every big system will end in a tangled mess. It doesn't matter whether the primitives are objects or functions. What are the problems I'm trying to solve? Undesired mutation of state. Why do I have to use classes? Because state mutates over time in a system where classes rely on each others state. But classical OO systems don't have the notion of time. Rx in .NET is a good try at solving this problem.

      Why do I have to run plenty of processes in parallel? I'm tackling complexity by not relying on objects that mutate state. I'm relying on functions that ask to use state as input. So where is this state persisted? How does git manage to be so good at managing state?