Prelude
“It was the summer of 2009.” — that’s how I would have started this article. I would have spent significant energy in coming up with a few catchy paragraphs after that, before getting to the main point.
However, it is the summer of 2023, and, thanks to the advances in what some people used to call machine learning, others black magic, and now collectively agreeing upon as “Artificial Intelligence”, a sizable portion of the human brain is now freed up to do other important things.
So, let’s start this article the post-modern way, and show the output of the following prompt to perplexity.ai.
Write a short story about the summer of 2009, and how a Java developer decided to bring Unix domain sockets to that realm.
Perplexity.ai writes, and I apologize profoundly for the length and content:
Once upon a time, in the summer of 2009, a Java developer named John was working on a project that required interprocess communication (IPC) between two processes running on the same machine. He was familiar with TCP/IP sockets, but he wanted a faster and more efficient way to communicate between the processes.
One day, John stumbled upon Unix domain sockets, which are like TCP/IP sockets but used only for local IPC. He was intrigued by the idea and decided to bring Unix domain sockets to the Java realm.
At first, John struggled to find a way to implement Unix domain sockets in Java. He searched online and found some libraries that claimed to support Unix domain sockets, but they were either outdated or didn’t work as expected.
Determined to find a solution, John decided to write his own library for Unix domain sockets in Java. He spent countless hours reading the Unix domain sockets documentation and experimenting with different approaches.
Finally, after several weeks of hard work, John was able to create a working implementation of Unix domain sockets in Java. He called his library “junixsocket” and released it as an open-source project on GitHub.
Over time, junixsocket gained popularity among Java developers who needed fast and efficient IPC between processes running on the same machine. Today, junixsocket is widely used in various Java applications, from web servers to desktop applications. Thanks to John’s hard work and dedication, Java developers can now enjoy the benefits of Unix domain sockets in their applications.
Well, that was entertaining. OK, I know, for a fact, that John didn’t write junixsocket (but kudos for getting the name of the library right!) Let’s refine the prompt accordingly.
Perplexity.ai, read my mind:
Please,
Write a short story about the summer of 2009, and how a Java developer named Christian decided to bring Unix domain sockets to that realm.
And the summoned did as I said:
Once upon a time, in the summer of 2009, Christian was a Java developer working on a project that required communication between different processes on the same system. He was frustrated with the limitations of TCP/IP sockets, which were the only option available at the time. He wanted a more efficient and secure way to communicate between processes on the same system.
One day, Christian stumbled upon Unix domain sockets, a type of socket that allows communication between processes on the same system using the file system as the transport mechanism. He was excited about the possibilities and decided to bring Unix domain sockets to the Java realm.
Christian spent the summer of 2009 working on a library that would allow Java developers to use Unix domain sockets in their applications. He faced many challenges along the way, including the lack of support for Unix domain sockets in the Java standard library and the need to write native code to interface with the Unix domain socket API.
Despite these challenges, Christian persevered and eventually released his library to the world. It quickly gained popularity among Java developers who were looking for a more efficient and secure way to communicate between processes on the same system.
Years later, in Java 16, Unix domain socket channels were finally added to the Java standard library, making Christian’s library obsolete. However, his contribution to the Java community will always be remembered as an important step towards more efficient and secure inter-process communication in Java.
Welp, that was a bit too close to hurting my feelings; “obsolete” is a harsh word.
Admittedly, some people may indeed see it so, but I am not here to indulge you with schmaltzy nostalgia about the good olden days where all we had was JNI and some brittle native C code.
Inarguably, it is the summer of 2023 now, even Java 16 is a thing of the past, and machines have made writing articles so much more, let’s say, streamlined. And one can guess that, give it a few more summers, machines will be really good at writing Java libraries, too.
One last chance, oh stochastic parrot that thou art in the cloud, reveal thy words, hast thou considered the following prompt:
Write a short story about the summer of 2023, and how a software engineer named Christian decided to keep working on this Java library for Unix domain sockets (which he started developing in 2009), despite being made obsolete with the introduction of unix domain socket support in Java 16.
And so, the parrot twitters:
Once upon a time, in the summer of 2023, there was a software engineer named Christian. Christian had been working on a Java library for Unix domain sockets since 2009. He had put in countless hours of effort and dedication into developing this library, and it had become his passion project.
However, with the introduction of Unix domain socket support in Java 16, Christian’s library had become obsolete. The new built-in support provided all the functionality that his library offered, and it was now much easier for developers to work with Unix domain sockets in Java.
Christian was faced with a dilemma. He could simply abandon his library and move on to other projects, but he couldn’t bring himself to do that. He had poured so much of his time and energy into this library, and he believed that it still had value, even if it was no longer necessary.
Instead of giving up, Christian decided to continue working on his Java library for Unix domain sockets. He saw an opportunity to improve upon the existing functionality and make it even more powerful and user-friendly. He wanted to create a library that would provide additional features and capabilities beyond what the built-in support offered.
Christian spent the summer of 2023 tirelessly working on his library. He added new features, optimized the code, and made it compatible with the latest versions of Java. He also reached out to the developer community, seeking feedback and suggestions for improvement.
Despite the fact that his library was no longer essential, Christian’s dedication and passion for his work never wavered. He believed that there would always be developers who would appreciate the extra functionality and flexibility that his library provided.
As the summer came to an end, Christian released the latest version of his Java library for Unix domain sockets. It received positive feedback from the developer community, and many developers found value in the additional features it offered.
Christian’s decision to continue working on his library, despite it being made obsolete, taught him an important lesson about perseverance and following his passion. He realized that sometimes, even if something becomes unnecessary or outdated, there can still be value in pushing forward and striving for improvement.
And so, Christian’s journey as a software engineer continued, fueled by his unwavering dedication and love for his craft. He knew that there would always be new challenges to tackle and new opportunities to explore, and he was ready to embrace them all.
Perplexity.ai, thou hast understood me well.
Motivation
You have reached the limit for Perplexity-generated perplexities, not the end of the article.
Yes, I had indeed started working on junixsocket in 2009. Mostly, because I wanted to use Java RMI between Java processes on the same machine. I had built a service-oriented architecture for the reboot of my news search engine, NewsClub. I got it working to the point where it worked for me, and perhaps some others, so I felt it should be released to the world as open source, giving back a little of what I took to use elsewhere in my codebase. And so it remained online as it was.
Years go by. NewsClub never restarted, for reasons beyond the scope of this article. I sought employment at-will, and certain terms of employment, although of questionable chance of enforcement, were imposed upon me as I continued my career. They made it clear that I’d better stay off developing open source software in my spare time.
Now that I am unshackled again, again for reasons beyond this article but under my control, I decided to double-down on my journey of dedication and love for the craft.
I also learned that, no, Java 16’s introduction of support for Unix domain sockets isn’t remotely signaling the end for junixsocket…
Interoperability with existing code
For one, the new Java 16 API only provides SocketChannel/NIO access, not Socket, and only streams, no datagrams. It also makes integrating the new abilities into existing code extremely difficult: Most Java code that deals with “Internet” sockets, such as Java RMI, works with InetSocketAddress-es, which were designed for IP (TCP, UDP, etc.) addressing and communication. The Java API creators took great care of encapsulating the already encapsulating BSD socket API even further. It was tailored towards IP communication.
junixsocket works with these idiosyncracies, allowing for a very smooth transition to Unix domain sockets for existing code.
Because of junixsocket’s design, we can, for example, easily reuse code for HTTP servers and clients (using a variety of implementations, even including Jetty, and serve content over UNIX domain sockets (or “AF_UNIX” with AF standing for address family). We can also use it to connect from Java (or any other JVM language) to a PostgreSQL or MySQL server over the socket file in the file system, instead of relying on local TCP/IP communication that could open up a security hole.
We can also make use of features not supported by Java 16’s native support of Unix domain sockets, such as passing file descriptors from one process to another. Remember, file descriptors can be anything, like referencing another socket, potentially created in a process with elevated privileges, being passed down to one that is more restricted.
Interoperability with other socket domains
Now, for the second important point. Unix domain sockets aren’t the only “Unix sockets” (BSD sockets, POSIX sockets, how you want to call them).
There are very interesting protocols out there that really wait to be made accessible from a high-level language like Java, for example: VSOCK (communication between a VM, its host and maybe its peers), TIPC (high-availability communication in a cluster environment), and even things like AF_SYSTEM on macOS, which allows talking to kernel-level features like creating VPNs (with junixsocket, a Java-only WireGuard implementation on macOS may be just around the corner).
junixsocket exposes these protocols in a way that feels like the proper Java-way.
For example, with junixsocket it’s trivial to serve websites over TIPC, in other words building your own high-availability HTTP cluster has never been easier.
Interoperability with different platforms
This last point is critical. We could have just exposed the low-level BSD socket API as-is, but you would be in for a bag of surprises if it wasn’t for junixsocket to shield you from the specifics of how certain operating systems interpret these APIs.
I’ve taken great care of building a suite of integration tests (“junixsocket-selftest”) that can be run on any supported platform, helping me find the rough edges of a system. I’ve also added a set of “capabilities” that a developer can check against to see if these are supported in the environment their code runs in. Both measures help determine the suitability of junixsocket before one actually runs into a problem while in production.
As a recent example, thanks to junixsocket-selftest, we were able to detect three bugs in the Haiku operating system, one being a bug around socket addresses, one around the “buffer-full” condition, and even one being a use-after-free bug that crashed the kernel.
I’ve also ventured into supporting mainstream and niche platforms alike, such as Linux, macOS and the BSDs (FreeBSD, OpenBSD, NetBSD, DragonFlyBSD), Solaris/OpenIndiana, Windows, Android, GraalVM, IBM’s operating systems AIX, IBM i, z/OS, and perhaps even z/TPF (for TPF, I have a working binary but no system to test with…). They all have their delicate intricacies, and while it was fun building support for them, it’s something someone else should be able to ignore and take for granted.
Lastly, junixsocket runs with Java 8 or newer. So even if you’re stuck in the past, junixsocket will help you out.
How the Sausage is Made
Realizing that this article already has a lot of words, today I want to spare you from the details of how all this is put together; such prose is for another day, unless you want to add junixsocket as a project dependency, dive right into the source code or ask the perplexing oracle.
I admit I enjoyed building junixsocket not only for the why and what, but also for the how of all this fitting together, for the craft itself.
What I will do today in this article is to stay high-level; further details, only sketched below, warrant their own blog posts. I will update this article with links once they are published.
The junixsocket package builds entirely on my Apple Silicon MacBook Pro, cross-compiling most supported target platforms out-of-the-box (as of version 2.7.0 twenty-four platform/architecture-specific binaries are included).
The build process, which is mostly based on Maven, constructs Java multirelease jars (with certain optimizations only available for newer Java versions), in probably the only sane and Eclipse-compatible way I’ve seen so far, supplying the JNI library without much further ado as a jar in the classpath/modulepath, Jigsaw-aware and all.
During that build time, numerous static code checkers, such as Checkstyle, Spotbugs and PMD (and occasionally, Google’s errorprone and Facebook’s infer) regularly tell me how stupid I am, before making me fix the code, and automatically reformat everything so neatly that bikeshedding in code reviews, even as soliloquy, becomes practically impossible.
Furthermore, I won’t bother you (yet) about how junixsocket appears to be faster than Java 16’s API in some cases (I don’t yet know why), or how JNI can be useful even in the days of Project Panama, which junixsocket is certainly going to explore in a future version.
A Shameless Plug
There is no paywall, no monetary obligation; junixsocket is Apache 2.0 licensed open source. So what am I asking for?
Having just released junixsocket version 2.7.0, I would like you to try it out, maybe again, and learn its many uses that reveal themselves only if you are truly invested in this excitingly deep niche interest.
Please visit the junixsocket website and have a look at its public repository on GitHub. If you have questions, don’t hesitate asking them.
If you find junixsocket is not useful for you, let me still thank you for allowing me to share with you the knowledge that this library exists.
Nervertheless, perhaps you find that junixsocket lacks a feature that you (or Perplexity.ai) would like to see included?
If so, give me a holler, send me a packet of demands over your favorite protocol.
Yours connectedly,
Christian Kohlschütter