The jank programming language
]( https://avatars.githubusercontent.com/u/1057635 )
Intro
Hi everyone! I’m Jeaye Wilkerson, creator of the jank programming language. jank is a dialect of Clojure which I’ve been working on for around eight years now. To start with, jank was an exploration of what my ideal programming language would be. Coming from a background of systems programming and game development, I was very comfortable with C++, but I also was exploring the world of functional programming. All of this led me to try Clojure, which ultimately refined the direction of jank altogether.
My work on jank is made possible by Clang, LLVM, and the tools coming out of the Compiler Research Group, including Cling/Clang REPL and CppInterOp.
The past two years of jank development have been exciting as I’ve focused on building jank into a native Clojure dialect on LLVM. This marries the lispy world of REPL-driven, data-oriented, interactive programming and the C++ world of light, fast, native runtimes. jank isn’t production ready yet, but it’s come a long way in the past two years.
What is Clojure?
Clojure is a modern, practical take on Lisp. It’s functional-first, has persistent, immutable data structures by default, but allows for adhoc side effects. As it’s a lisp, it has a powerful macro system which allows devs to transform their own code using the same functions they use to transform any other data. Clojure is dynamically typed and also garbage collected.
Clojure has a rich tooling story for interactive development. Clojure devs generally connect their text editor directly to their running programs, via a REPL (read, eval, print, loop) client/server. This allows us to send code from our editor to our program to be evaluated; the results come back as data which we can then continue to work on in our editor. This whole workflow allows any function to be redefined at any time during development and it allows for a development iteration loop like nothing else.
Clojure is a hosted language, which means it has a symbiotic relationship with the host environment on which it runs. By default, Clojure is on the JVM. Clojure JVM enables seamless interop with other JVM languages like Java, Kotlin, Scala, etc. However, Clojure runs on many other hosts. For example, ClojureScript runs on JavaScript (browser or node). ClojureCLR runs on the CLR (where C#, F#, etc run). There’s also ClojureDart, ClojErl (BEAM), and many others. Each of these dialects provides seamless interop with its host. Since Clojure is always hosted, things like socket libs, filesystem libs, etc are never included in Clojure; they just come from the host.
Clojure on a native host, though, is where jank comes in.
What makes jank special?
jank provides all of the interactive functionality of Clojure, which involves JIT compiling native code with LLVM, while also providing seamless interop with the native world. For those who are familiar, jank for Clojure is basically analogous to Clasp for Common Lisp. However, compared to Clasp, jank aims to provide even more seamless native interop which doesn’t require making “bridge” files.
Interop features
In jank, when you require a module (like including a header in C++ or importing a module in Python), that module may be backed by a jank file or by a C++ file. If it’s backed by a C++ file, jank will JIT compile that file and then bootstrap it by invoking a well-known function. This allows you to vender C++ code alongside your jank code within the same project seamlessly.
For example, take a look at this jank file which requires both a jank dependency
and a C++ dependency. They’re both required in the same way. When
clojure.pprint is required, jank will find that the module is backed by a
.jank file, so it will JIT compile that file. However, when sleep-native is
required, jank will find that it’s backed by a C++ file and it will JIT compile
that using clang::Interpreter. In either of these cases, if a .o is present
and up to date, jank will load that instead.
(ns nap-time
(:require [clojure.pprint :as p] ; JIT loads a jank module
[sleep-native :as s])) ; JIT loads a C++ module
(defn main []
(s/sleep 500)
(p/pprint "napped!"))
The sleep-native module can be backed by a C++ file which looks something like
this. jank will look for a special jank_load_sleep_native function in the
module, based on the name of the module itself. Note, the -native suffix is
just a common pattern used for modules backed by C++ files.
object_ptr jank_load_sleep_native()
{
/* Create the sleep-native namespace so it's exposed to the jank world. */
auto const ns{ _rt_ctx->intern_ns("sleep-native") };
auto const intern_fn{ [=](native_persistent_string const &name, auto const fn) {
auto const boxed_fn{ make_box<obj::native_function_wrapper>(fn) };
ns->intern_var(name)->bind_root(boxed_fn);
} };
/* Create a var holding a pointer to the sleep function. */
intern_fn("sleep", [](object_ptr const ms) -> object_ptr {
auto const duration(std::chrono::milliseconds(to_int(ms)));
std::this_thread::sleep_for(duration);
return nil_const;
});
return nil_const;
}
Overall, this automatic module support allows for arbitrarily complex C++ files to be loaded, which will generally be used to wrap some existing native code and use jank’s C or C++ API to expose that code to the rest of the jank system. However, this is all still roughly equivalent to “bridge files” and I aim to do better.
On top of that, jank will also support interop directly from jank code into C++ land, allowing for the instantiation of native C++ objects as locals, calling native functions, and also instantiating C++ templates on the fly. This is where CppInterOp comes in. At this point, it’s only in design phase, but I aim to utilize CppInterOp to its fullest so that we can allow for arbitrary C++ values within jank, calling C++ functions, and working with C++ data types. The working design looks something like this:
; Feed some C++ into Clang so we can start working on it.
; Including files can also be done in a similar way.
(c++/declare "struct person{ std::string name; };")
; `let` is a Clojure construct, but `c++/person.` creates a value
; of the `person` struct we just defined above, in automatic memory.
(let [s (c++/person. "sally siu")
; We can then access structs using Clojure's normal interop syntax.
n (.-name s)
; We can call member functions on native values, too.
; Here we call std::string::size on the name member.
l (.size n)]
; When we try to gives these native values to `println`, jank will
; detect that they need boxing and will automatically find a
; conversion function from their native type to jank's boxed
; `object_ptr` type. If such a function doesn't exist, the
; jank compiler fails with a type error.
(println n l))
All of this will work with RAII, allow auto-boxing, but also give complete access to C++ from within jank.
AOT features
With jank, you can AOT compile your programs to two different runtimes:
- A dynamic runtime which enables REPL usage, further JIT compilation, redefining functions, etc
- A static runtime which strips out all JIT capabilities, enables LTO, and directly links functions instead of going through vars
Static runtime binaries will also be able to be statically linked, providing a portable binary which will very easily run on any distro or in any docker container.
Why would people use jank?
There are two main audiences for jank.
Clojure users
jank’s runtime is significantly lighter than the JVM. Since its runtime is specifically crafted for jank, rather than being a generic VM, it’s also able to compete with the JVM in terms of runtime performance while keeping memory usage lower.
On top of that, anyone who is using Clojure and wanting easier access to native libraries, tighter AOT compilation, etc, without sacrificing the fundamental Clojure interactive development workflows will choose jank.
Native users
In game development, where my career has been focused, we build our engines and games in C++, generally. However, we very often include a scripting language like Lua in the engine so that we don’t need to write all of our game logic in C++. jank is an excellent fit here, too. Not only does it provide seamless interop with the existing C++ engine/game code, but the REPL-driven workflows it supports will allow you to spin up your game once and then send code to it, get data back, and continue iterating on the code and data shapes without ever restarting your game.
Naturally, this is a game development focused use case, but the same thing applies to any existing native code base which wants to utilize a higher level language for a tighter iteration loop.
Next steps
jank is not yet released in production, but I aim to have it out the door in 2025. In fact, as of January 2025, I’ll have quit my job at Electronic Arts to focus on jank full-time, unpaid aside from some small sponsors. I’m aiming to get jank released, gather initial feedback, and help existing Clojure users onboard jank into their systems.
The three big tasks that are remaining before jank is released are:
- Improved error handling (following the recent work on Elm, Rust, and others)
- Finalized C++ interop support (using the CppInterOp library)
- Finalized AOT compilation support
Future plans
In the long term, since I’m building my dream language, I won’t stop at just a native Clojure dialect. jank will always be able to just be that, but I’ll also be extending it to support gradual typing (maybe linear typing), explicit memory management, stronger pattern matching, value-based errors, and more. This will allow another axis of control, where some parts of the program can remain entirely dynamic and garbage collected while others are thoroughly controlled and better optimized. That’s exactly the control I want when programming.
Thank you!
For everyone who has worked on Cling, Clang REPL, and CppInterOp: Thank you! jank is possible because of the magic you folks have created. To Vassil, in particular, thanks for all of the support over the past couple of years as I onboarded with Cling, ported to Clang, and then ultimately picked up CppInterOp as a means to tackle seamless C++ interop.
