So the Linux update sure is taking longer than planned.
Some part of the issue was with scheduling and work load, the last couple of months being particularly busy. But the real problem is that we were treading uncharted waters with the Linux release and it caused us to effectively spin our wheels in place some of the time.
So when an approach is not working, you can keep at it and see where it leads or you can try something new. We’ll try the later and hopefully this will help things. So no longer will we dedicate most of the effort to building a single working Linux version to the detriment of the compiler development and instead we will set the Linux port as a background task, to be finished and released as soon as it is done, but in the meantime continue a good release schedule for the compiler and standard library.
Second, we realized that we will never be able to cover a meaningful subset of Linux distributions with official releases so most of the Linux development effort was centered around making the compiler open-source, so that anybody could compile for the rest of the platforms if a version was not available. Currently, the compiler is mostly written in C++, but some parts are written in Z2. The bridge between them is a horrible collection of hacks. So we attempted to port everything to Z2 and release only that, with no hacks. This meant not only porting the compiler, but a whole lot of support C++ library. This was far too much of a substantial task to finish in any decent time period.
But actually this was a bit of a self defeating effort. Z2 is designed with two backends in mind, a C++ one and a LLVM one, and the C++ can offer excellent comparability, allowing both Z2 to be called form C++ code and they other way around (with some limits of course). So we will focus the effort on making the natural compatibility closer to production ready, rather than porting. After this, new parts of code will be written only in Z2 and every time a bit of C++ code gets some non-trivial update, we will rewrite that bit of code in Z2, one function at a time. And thanks to the designed comparability, this should work without any of the hacks that are currently needed. Slowly, over time, the number of Z2 functions will go up and the C++ one will go down, but in the meantime builds will continue to be hybrid. This way we don’t need to wait until we have a full port that we can open-source. Every single future version will come with the parts that are written in Z2 included in the package.
But having everything OSSed is only the beginning. This won’t really help potential users because as a fledgling language, there is ultimately an almost 0% chance of getting included into any Linux distribution. So you will have to compile yourself on all platforms that are not officially supported. And it won’t be that simple, since bootstrapping is needed. But Linux users willing to try a new programming language should have little problems with the process.
With an ever increasing projected amount of Z2 code in the compiler, it will soon come to us having to fix develop, fix bugs and debug Z2 code in a real project, the compiler itself. Doing this in C++ is easy because the tools are there and work well. Doing it in Z2 is harder, so ZIDE and the rest of the tools must become top notch. Particularly the current SCU output method, where methods are reorganized into a single file in an order that is best for the compiler but not that easy to use by a human won’t do. We’ll keep this method around, but a new more natural and traditional unit based compilation process will be introduced.
This will be a challenge since the goal is for Z2 to compile a faster than C++.
In January we want to release PT 9.2, a version that will support this new compilation method. It will also have a higher percentage of Z2 code in the compiler, this time OSSed. The library will get some additions. We also noticed that it would be possible to improve in some areas on one of the core tenets of the language: in general, there should be one way/syntax of doing a single thing, and if there are more than one, this should be particularly well explained and documented. The language is still not as lean and mean as we would like it, so some parts are getting removed. This should have pretty much zero impact on the library and the capabilities of the language, it will just become less redundant and have a more focused syntax. Part of these updates will be included in 9.2, including the new much improved and less ambiguous way of using vector literals.
Once this process is fully finished, we will release further PT 9.x version. These will have higher meta-programming capabilities and some other features. PT 10 should bring full Linux support and before 11, all source code, including the compiler should finally be OSSed.
Over the holidays I’ll try to write a new Crash Course. I am also experimenting on a larger written piece, titled “Language specs by minigun”, where I’ll try to present the language using short phrases and examples, fired at a break-neck speed. Summarizing pages form the spec into a couple of lines at a time. And hopefully once that piece is done, it will go though the hands of an editor so we can improve on the generally low level of written English on this blog.