Class constructor performance foibles?

So, while working on PT 9.1, the SystemCPU class was added:

class SystemCPU {
	val Vendor = "";
	
	val MMX = false;
	val SSE = false;
	val SSE2 = false;
	val SSE3 = false;
	val SSSE3 = false;
	val SSE41 = false;
	val SSE42 = false;
}

The class isn’t complete yet since it is missing some options, including core count and the brand name for the CPU. But even when it going to be finished, it is going to be simple class since it has no functionality. The System class will instantiate one of these for you and fill it up with the appropriate information so you have a platform independent way of figuring out the capabilities of you CPU and things like number of cores and what-not.

So this is and is meant to be a simple class. It has a String field and several Bool fields. As it turns out, the current implementation of String causes its fields to be zeroed and false is also 0, so what a constructor for SystemCPU must do is set everything to 0. Do basically a memset(0). Since he have the C++ backend, one can always output pretty readable equivalent C++ code, and here are the constructors for String and SystemCPU:

String::String() {
	data = 0;
	length = 0;
	capacity = 0;
}

SystemCPU::SystemCPU(): Vendor(_Void) {
	new (&this->Vendor) ::String();
	MMX = false;
	SSE = false;
	SSE2 = false;
	SSE3 = false;
	SSSE3 = false;
	SSE41 = false;
	SSE42 = false;
}

The Vendor(_Void) and new (&this->Vendor) ::String() are a bit weird. The sequence Vendor(_Void) guarantees that the C++ SystemCPU constructor will leave Vendor alone (the optimizer recognizes the Vendor(_Void) construct due to a special constructor in String and generates a NOP for that). With the placement new, new (&this->Vendor) ::String(), we manually call the constructor for String and then initialize the rest of the fields. And all fields get initialized to 0.

The question is: should we optimize this initialization? When working with the C++ backend, we always need to decide if we do an optimization in the front-end or leave-it to the backend. Leaving it to the backend can result in more readable code. But does the backend do this? Let’s benchmark!

using sys.core.StopWatch;
using sys.core.SystemCPU;

class BenchSystemCPU {
	const TIMES = 100000;
	
	static def test_Abs(const in: [c]SystemCPU) {
		for (val i = 0; i < TIMES; i++) {
			for (val j = 0p; j < in.Length; j++)
				in[j]{};
		}
	}

	static val buffa: [c 100000]SystemCPU = void;
	
	def @main() {
		{
			val sw = StopWatch{};
			test_Abs(buffa);
			System.Out << test_Abs.Name << " finished in " << sw.Elapsed() / 1000 << " sec.\n";
		}
	}
}

This simple little benchmark will fill up a statically allocated vector of 100000 SystemCPU instances 100000 times. Never-mind that the program only due to the sheer coincidence that everything is initialized to zero doesn’t leak memory. In order for it to not leak memory the call of a manual constructor must be accompanied by a call to manual destructor too, but calling the destructor would detract from the goal of the benchmark since the destructor is not free. And even if it were to leak, we are benchmarking the time it takes to call the constructor, so it is not important.

So here are the results:

  • 37.1421495178437 seconds on MSC14, 32-bit, release mode.
  • 35.1300343545186 seconds on TDM, 32-bit, release mode.

Fair enough! But what if I am wrong about the _Void bit and the placement new. What if these two constructs completely confuse both compilers? Well, we can easily rewrite the two constructors to be much more standard:

String::String(): data(0), length(0), capacity(0) {
}

SystemCPU::SystemCPU(): Vendor(), MMX(false), SSE(false), SSE2(false), SSE3(false),
						SSSE3(false), SSE41(false), SSE42(false) {
}

Now all fields are initialized in the initializer list and I also followed the declaration order. Let’s see the benchmarks:

  • 37.1537800874641 seconds on MSC14, 32-bit, release mode.
  • 35.1234764643324 seconds on TDM, 32-bit, release mode.

The results are the same, barring a bit of standard variance, so the constructs are not confusing the compiler.

But we are 600 words in and you may be asking the following question: what is this pointless benchmarking all about. Well, I’ll do one final tweak and replace the initializer list with a memset(0). It is a bit hacky and not as pretty or maintainable, but one would except the compiler to actually do this behind the scenes and if we get the same numbers, then that is evidence enough that the memset hack should not be used. Here are the modified constructors:

String::String() {
	memset(this, 0, sizeof(String));
}

SystemCPU::SystemCPU(): Vendor(_Void) {
	memset(this, 0, sizeof(SystemCPU));
}

And the results:

  • 8.30797196732857 seconds on MSC14, 32-bit, release mode.
  • 16.5283915465819 seconds on TDM, 32-bit, release mode.

This is insane! The MSC version is roughly 4 times faster and the TDM version is roughly 2 times faster with this hackjob. To investigate the 4 vs. 2 times difference I would need to go into the assembly and into memset and see what is going on, but that is not the point.

The point is: are C++ compilers not equipped with an optimization pass to handle this? Because if they are not, adding such a pass to the front-end would be a huge win.

This issue needs further investigation and I’ll be back with a follow up after I dig though some ASM. Hopefully it is not just some mistake on my part!

Advertisements

Post PT 9.0 Updates and OSS

Z2C PT 9.0 was announced here and on Reddit last week and released as a pre-alpha last week. So the question is: what now?

Development on PT 9.1 has started. It will contain bug-fixes, new features and library additions. But this goes for all versions from now on until the implementation is considered done, so I won’t repeat this every week. Instead I’ll talk about long term goals.

One of the Reddit questions was: why not release the source code as OSS. Now, the standard library is released as OSS, but the compiler itself and ZIDE are not. The reason I gave for this was time an perfectionism. So everything will be released as OSS. The compiler itself will be included in the standard library. Thus, it must look decent and have a stable API. The code is no where near ready for this and in consequence it is not OSSed yet.

A lot of time is needed to achieve this and in order to compensate for this, the compiler library will be split up more. A part of it will be separated as a low dependency “scanner”, a library that goes over Z2 code and builds class layout and other meta-information. This new library shall work without the compiler or the assembly and is designed to be very lenient with errors if needed, in order to provide all the code navigation features for ZIDE and other tools. So step one is to do this isolation part and refactor it a bit. This small library will be called “z2c-scan” and will be the first to be OSS. Using this, ZIDE can be OSSed as well, since it has no other dependencies.

The rest of the library will be split up again, with z2c-comp, z2-cpp and z2c-llvm being the compiler, C++ backend and LLVM backend respectively.

And a very long term refactoring process will be executed incrementally on all the code to make it fit for prime time and not leave it as ugly and dense as it can be in some places. In the early stages of the compiler, there was a goal to get it as small as possible. A pretty stupid goal and in consequence the compiler code is very dense and not that easy to maintain. This will change with time.

Other than that, there will be an update every two weeks. Even if these smaller updates are light on features, they do contain bug-fixes and will in turn create a better user experience.

Z2 PT 9.0 Available for Download

Here it is, our first pre-alpha release, just on time!

Well, one day latter, since when setting the release date, nobody checked to see what day it was and the 15th of August is a Holiday. No problem, the 16th is just as fine. And this isn’t even the only mistake related to this release: turns out that WordPress.com doesn’t allow you to host ZIP files and make them available for public download. So the release package will have a temporary home until a more permanent location is found:
Download z2-pt-9.0.zip for Windows 32.

As detailed in a previous post, this first release is Windows only. Future releases will be Linux too. Sorry!

So what is the status of the release? The estimation puts it 3-4 weeks behind where it was planned to be, but the delay doesn’t effect it that much. It is pretty stable all things considered. And do note, this is a pre-alpha, so don’t expect the compiler or ZIDE to be production ready. A bug-fixing patch is planned 2 weeks from now.

The delay effected the compiler and there are two known bugs in this release. For starters, aliases are so bugged that they were disabled them. Aliases are a very useful feature, but more widely used with larger projects. Our small scale testing didn’t show the bugs until it was too late. With the current implementation, there is no easy fix. What we need to do is replace this implementation with the class lookup mechanics we use everywhere else in the compiler. The reason this was not done yet because this class lookup code is scheduled for an update for PT 9.1, so working on it now would have been in wain. The second bug is related to function overloading. Last time we refactored function overloading, it was believed to be sufficient and handle all cases. Turns out there are still some issues. This will take some time to fix, so either PT 9.1 or 9.2.

The standard library came out pretty much as planned, but the two above bugs prohibited us from including the Color classes. They exhibit both the overloading bug and use aliases. There wasn’t enough time to include 3 more container classes. Including Vector, the most commonly used of them all. Scheduled for 9.1.

ZIDE was barely effected by the delay. Maybe some minor polish could have been performed, especially related to directory renaming, but it is fine for now. PT 9.1 will introduce a very early code-completion feature.

That’s about it for this post. A new post, or even better, a static page, detailing what the package contains and how to use ZIDE is in the works. That page will then be used as reference for all releases from now on so we don’t have to repeat basic instructions.

Developer Preview on August 15th

With this project it is our goal not just to create a programming language and a standard library for it, but also to do a lot of experimenting, to figure out what works best and why. This is why we followed up upon a very interesting and aggressive optimization technique for containers. You couldn’t apply it in all cases, but when you could, it was pretty much the fastest way possible of getting a new instance into a container. We were pretty sure we were onto something quite valuable here but… in the end it turned out to no work at all well with exceptions. Now, the technique is not dead and buried. It can still be used when we detect that the code can’t throw, but this pushes the feature from a valuable top tier optimization to a niche optimization you may or may not do when implementing the last features of an otherwise mature compiler.

The moral of the story is that at this stage we can’t afford to spend time on such features. We are 2 months behind schedule and there are 3 draft posts that are just waiting there for their final edits in order to get published. Additionally, because of our perfectionism, we didn’t manage to commit anything to GIT that is not related to UT. So we need to refocus and get something out there.

We will release a pre-alpha package labeled a Developer Preview on the date of August 15th. A set date is needed in order to focus the development process on high yield areas, not niche optimization. But the date is so close that this Developer Preview will not be as ambitious as it could. Future previews will be more substantial, but this one will have and intentionally reduced feature set and its goal is to be a working downloadable and testable prototype. I will now detail all the corners that will be cut in order to make this deadline.

 

The focus will be on the language

We could try to release the language as complete as stable as possible at the expense of the standard library. Or we could focus on the standard library, since the language is in a pretty good shape right now. Or maybe somewhere in the middle.

It was decided that for the first preview, the focus will be on language. During the next month, the language and the compiler will receive most of the focus and the library will only get small fixes and maybe a couple of new methods here and there.

 

Only the C++ backend will be supported

As you may know, the compiler is designed with a flexible backends and the plan is for it to eventually ship with 3 backends: LLVM, C++ and C. The LLVM is planned to be developed only after the C and C++ ones, so it was of course not going to be included in the first release, but we decided to also exclude the C one. It is significantly easier to polish a single backend per release and it will probably end up being a higher quality one this way. We are not 100% sure yet, but probably the C one will be pushed back after the LLVM completion.

We’ve been polishing the C++ backend for two weeks now and the work has greatly payed off. The old backend was designed with several profiles in mind which all balance used language features, performance of code, performance of compilation and code aesthetics. There was a profile that generated quick and extremely ugly code. One which generated extremely beautiful code meant for C++ programs to call the Z2 standard library. The new backend has done away with this and now all generated code is decent looking and carefully selected to compile and behave the same on all supported compilers. There is a minimum level of aesthetics. And the code looks a lot more natural, using a more C++ style. And an additional library mode which goes the extra mile.

 

First version will only support a single OS

As with the multiple backends, focusing on just one OS will allow us to give that version extra polish. So the first version can be either Linux or Windows. Unfortunately here we don’t get to choose: most of the standard library is platform agnostic, but some system calls are made when printing to the console or using synchronization and these are currently implemented using Windows API. So the first version will target Windows only. And only 32 bit builds. Developer preview #2 will support 32 and 64 bit builds and the Linux version.

Just to be clear, Z2 is not tied in any way to Windows. It even works today on Linux. ZIDE is build using cross-platform open-source GUI, so there are zero issues with it. The compiler is again portable and indeed does compile 100% of the test-cases under Linux. Unfortunately, it will also fail to link-edit said binaries under Linux because of the WinAPI calls that need to be replaced for Linux. Were the release date two months form now, the first version would support Linux and Windows, but since it is just one month, only Windows will be included. In 2 to 4 weeks after developer preview #1 is out, both Windows/Unix support and 32/64 bit support will be finalized and stable. And after that we shall be targeting Mac too.

 

Support for compilers out of the box will be limited to MSC and GCC

The C++ backend of course requires a C++ compiler installed on your system. We believe that it is vital that Z2 works out of the box, so you will not be left to your own devices in invoking this compiler. Z2 will autodetect the following compilers:

  • TDM GCC
  • Visual Studio 2010/MSC10
  • Visual Studio 2012/MSC11
  • Visual Studio 2013/MSC12
  • Visual Studio 2015/MSC14

For GCC, TDM GCC will be included in the package so that you can compile out of the box. This is the default build method. For Visual Studio versions, the auto-detect should pick up both the commercial and the Express/Community editions. You do need to have the Platform SDK installed. If you can compile WinAPI programs with your installed compiler, you should be fine. It will not auto-detect Visual Studio “15”/MSC15, Microsoft’s own Visual Studio “developer preview”, because it is not out yet. It will be supported when it is out, granted it still supports traditional Windows executables. The are no plans to support Windows 10 styled universal apps (UWP).

The auto-detected methods are stored in the “buildMethods.xml” configuration file, the one file that is used by all the tools inside the Z2 package that need a compiler. So if your compiler is not auto-detected or you want to use the same version of GCC/MINGW/TDM that you have installed and normally use, you can go in and edit that XML.

 

Backends will only be used in SCU mode

Well, this one is actually not true. A bit of a white lie. First, let me explain what SCU is. It stands for Single Compilation Unit. Manual SCU is pretty difficult to use, but automatic SCU can have a lot of benefits. Initially, Z2 had only support for automatic SCU. Meanwhile, we transitioned to multiple CUs that are still automatic. This has more advantages. So what we mean by SCU mode is that it is automatic and you have no control over your compilation units. SCU must be retroactively made to stand for “Smart Compilation Units”.

Now, in a more mature Z2 compiler you can optimally have full control of your compilation units and deactivate/fine tune the automatic system. This feature will not be supported in dev preview #1 and you will have to do with SCU.

 

There will be bugs!

In an ideal world, the package you get on 15th will have zero bugs. But that is highly unlikely. We’ll fix as many bugs we can, but some will get though. Also, some language features might not make it in their final form. There will probably be very few breaking changes in the future, but this is not a guarantee. Developer preview #1 is pre-alpha after all!

CrashCourse – 006 – Templates and relationals

Last time I introduced the Intrinsic class together with some handy relational operator related functionality. But since Intrinsic is now the home of operations like Clamp and Max and these operations are defined by templates, without filtering the template parameters they can receive, anything can be passed to these functions. Even things that are not comparable!

So let’s see what problems this can cause and how to solve them. Let us consider a simple Version class that holds the major, minor and revision number of some product. A very simple class, not meant to be functional, just a simple example:

class Version {
	val major = 0;
	val minor = 0;
	val revision = 0;
	
	this(maj: Int, min: Int, rev: Int) {
		major = maj;
		minor = min;
		revision = rev;
	}
}

To break up the monotony of large blocks of text on the blog, I shall show a screenshot of the typical ZIDE workflow that is used when developing the samples for this blog and anything else Z2 related:

006less01

A quick tour of the sample. On line 3 we define our simple Version class and on line 15 we define a class with a @main slot method to test the version class. On lines 17 and 18 we instantiate two version variables.

On line 20, we test the equality of these two variables. As described in CrashCourse – 003 – What you get for free, Z2 will figure out common task for you from a small pool of common tasks. Simple straightforward comparison of value types is one of these tasks. The compiler takes one look at the Version and has zero problems to test equality of two instances. You as a programmer you shouldn’t have to write such easy boring code that only compares 3 Ints. Same for line 21, where we test for inequality. These two lines work out of the box because aggregate value type equality if a well defined unambiguous operation. Such methods that are provided automatically by the compiler for you are called automatic methods. They are always provided, but you can suppress this for a class/method combination if not needed.

And this is the reason why line 22 fails to compile. On this line, we don’t use operators == or !=, but operator <. The “less” operator can’t be defined unambiguously for aggregate types and the compiler can’t resolve v1 < v2. This is what the error says. Maybe the error message could be improved though.

The solution for this compilation error is to provide a < since the compiler can’t provide one for us. In Z2, method names that start with @ are called slots and they are just regular methods, but in some context the compiler will call them implicitly. Like @main. The calling mechanisms of the slots makes them perfect candidates for defining operators in classes. This was deemed a better solution than using a keyword like in other programming languages and the literal operator. It is easier to read, type and manually call. And solves some additional problems, like with pre and post ++ operators.

The < can be defined using the @less method:

class Version {
	val major = 0;
	val minor = 0;
	val revision = 0;
	
	this(maj: Int, min: Int, rev: Int) {
		major = maj;
		minor = min;
		revision = rev;
	}
	
	def @less(const v: Version): Bool; const {
		System.Out << "call " << class << '.' << def.Name << " of class " << @neq.class << "\n";
		if (major < v.major)
			return true;
		else if (major > v.major)
			return false;
		else  {
			if (minor < v.minor)
				return true;
			else if (minor > v.minor)
				return false;
			else
				return revision < v.revision;
		}
	}
}

This updated Version class defines on line 12 the @less slot. Now, when we do v1 < v2, the @less method will be called on the v1 instance and v2 will be passed in as a parameter. This is of course just syntactic sugar since @less is just a normal method with a name that starts with @, so it can be called normally: v1 < v2 and v1.@less(v2) are equivalent and result in the same machine code. The actual implementation is not important. I used here a simple implementation of the top of my head for comparing versions and may not be the optimal way to compare versions in production code. It is just a sample. This method could be implemented to not work as a relational less operator, even though it represents that slot. It is a very good idea not to do this to avoid major confusion.

One strange thing about this method is line 13. This is just for this sample to test that this method is actually called. Normally, you wouldn’t add such tests in real code. It could have been a simple System.Out << "Hey, @less has been called"; but I opted for this more complicated statement to demonstrate some reflection. Z2 has both compile and run time reflection and these features combined with templates and can be quite powerful. But in this sample we only use reflection for basic debugging. First we print out class. This is equivalent with this.class and returns the compile time class information for this, a reference to the current instance. def is similar to this, but does not represent the current instance, but instead the current method. Like class.Name, def.Name will return the name of the current method. And similarly to this.class, def.class returns the class information for the current method. In Z2 everything is an instance of a class, even methods. The class of all methods is Def. But instead of printing out def.class, I printed out the class of another method: @neq.class. It is the same class and I used this in the sample both to demonstrate that the classes are the same and that automatic methods are there and present, even if it not obvious that they are: @eq is the == operator and @neq is the != operator. So the output of this program after the fix will be:

false
true
call Version.@less of class Def
true

With @less working, now we can try a much more complicated sample, where we use all relational operators, Min,Max, Clamp and so on:

class Test {
	def @main() {
		val v1 = Version{1, 2, 7000};
		val v2 = Version{2, 0, 1};
		
		System.Out << (v1 < v2) << "\n";
		System.Out << (v1 > v2) << "\n";
		System.Out << (v1 <= v2) << "\n";
		System.Out << (v1 >= v2) << "\n";
		
		System.Out << "Min: " << Intrinsic.Min(v1, v2) << "\n";
		System.Out << "Max: " << Intrinsic.Max(v1, v2) << "\n";
		
		val v3 = Version{1, 0, 0};
		Intrinsic.Clamp(v3, v1, v2);
		System.Out << "Clamp: " << v3 << "\n";
		
		val v4 = Intrinsic.Clamped(Version{7, 5, 6}, v1, v2);
		System.Out << "Clamped: " << v4 << "\n";
	}
}

call Version.@less of class Def
true
call Version.@less of class Def
false
call Version.@less of class Def
true
call Version.@less of class Def
false
call Version.@less of class Def
Min: 1 2 7000
call Version.@less of class Def
Max: 2 0 1
call Version.@less of class Def
Clamp: 1 2 7000
call Version.@less of class Def
call Version.@less of class Def
Clamped: 2 0 1

The interesting part of this sample starts on lines 6-9. We only defined operator <, but we can use >, <= and >=. Think of it as the compiler making up for the fact that it couldn’t provide you with a free < operator. If you have @less defined, but not @more(the > operator), the compiler can still handle > by swapping the two operands: v1 > v2 is compiled as v2 < v1. And since we provided less and @eq, the equality operator, is an automatic method, the compiler can use them both to provide operator , called @lesseq and @moreeq. And once all these operators are defined manually or automatically, you can now use all the stuff from Intrinsic. And the same applies for when you have @more defined but not @less.

This part of the language design may be a bit confusing at first, so let me reiterate the rules:

  • @eq (==) and @neq (!=) are automatic. You get them for free, but you can of course override them and do something different. For POD types you rarely need to, but or non POD types and types that embed pointers, you may need to provide a better implementation since the automatic one might not do what you need.
  • @less (<) and @more (>) are not automatic. You need to define at least one of them! If you define both, they are use appropriately. If you define only one, their opposite is resolved by swapping around the operands.
  • @lesseq (<=) and @moreeq (>=) are automatic only if you defined at least one of @less and @more. Once you define at least one of these two, you get all 4. You are free to override @lesseq and @moreeq as with the others, and sometimes it is worth it from a performance point of view. To do <=, the compiler may need to do both < and =. It is sometimes possible to implement <= in a more efficient way.

So to reiterate, in order to get full relational operator coverage, you need to define either @less, @more or both!

In the git repository you can find these samples and more, part of the daily unit-testing.

With this, the very basics of the core numerical types are covered. Next time I’ll extend upon this as a jumping off point to introducing vectors!

CrashCourse – 005 – Int and Intrinsic

Last time I wrote about the basics of the Z2 library using an older and shorter version of the Int class. It is an archetypal value type and behaves similarly in a lot of languages, so it is easy to understand. I described how it handles conversions and operators using intrinsic functionality, how one can use constants to allow a class to offer some basic information about its value range and showed a few methods and properties.

The design looks viable, but has a few problems. It is easy to see this once you try to expand upon the library by adding a few more basic types. Just adding a single class, like Double, the only dependency of Int in this sample, would see us repeat the same code with minor changes. Defining the constants each time makes sense, since they have different values. But how about some methods? Like GetMin and GetMax? Sure, they are short and having them copied over into each class, including third-party classes is not a big issue, but surely there must be a better method.

This is where intrinsics come in! Last time we talked about two types of intrinsics: conversion constructors and operators, both in the context of numerical types. These represent the highest level of intrinsic functionality: they just exist and are part of their respective classes without any formal element to hint at their existence. But there are more traditional ways to access intrinsic functionality, with the main one being the Intrinsic class. This is a class only with static methods, offering a wide-set of common functionality. And this functionality is accessed using normal methods in a normal class, so it becomes easier to gain awareness of what is available.

Determining minimum and maximum values is an example of such functionality. Intrinsic.Min will return the minimum of the provided parameters. Instead of using:

5.GetMin(9);

…you now use the much more natural syntax of:

Intrinsic.Min(5, 9);

This approach has multiple advantages, beyond the already mentioned more natural syntax. It solves the problem of having to repeat the body of GetMin in each class. Intrinsic.Min is now a template method, so it only needs to be defined once and works with all types. Additionally, while some methods inside the Intrinsic class don’t have a visible implementation, Min does and it can be useful to see what it does. And finally, this method, and its counterpart, Max, is designed to work not on individual values, but value providers, so you will be able to pass it any combination of containers.

With this first change, we eliminated two methods not only from Int, but from all comparable value types from the library. What about Clamp? First, let us ignore Intrinsic and focus on naming conventions. During the development of the library we introduced a convention related to actions that can be applied to instances: these actions are implemented using verbs. A verb in its base form describes a mutating action, one that modifies the instance. A few examples: Add, Insert, Delete, Clamp, Sort and so on. Naturally, these methods can’t be called on const instances. Verbs using the past tense do the same thing as the base form action, but do not modify the instance, instead returning a new instance and leaving the original unchanged. The same examples: Added, Inserted, Deleted, Clamped, Sorted and so on. This is just a convention and there is no obligation for third-parties to respect it. So using this convention, in our Int class, we should have two methods. If the variable a is an Int with value 5, a.Clamp(10, 100); would modify a to be clamped to the range of 10-100, in this case making it have the value of 10, while a.Clamped(10, 100); would leave a as 5, but return 10. Additionally, a.Clamp(10, 100); and a = a.Clamped(10, 100); are equivalent. This holds as a general rule, with foo.Bar(); being equivalent to foo = foo.Bared();, but the former may or may not be more efficient, depending on what operator = does and the quality of the compiler’s optimizer.

So using this convention, our second version of Int would have two methods instead of one: Clamp and Clamped. Which leaves us with the same problem: two methods which are almost always the same, having to be copied over to a bunch of classes. Intrinsic solves this again, by having two methods, Clamp and Clamped:

class TestClamp {
	def @main() {
		val a = 5;
		Intrinsic.Clamp(a, 10, 100);
		
		System.Out << a << " " << Intrinsic.Clamped(-5, -100, -10) << "\n";
	}
}

10 -10

This solves the problem, but there is more to it. 0.GetMax(-1) wasn’t the most natural syntax, but a.Clamp(min, max) is. In some cases we want a class to have a “clamp” method independently from the Intrinsic class. We could just add the method to such classes, ignoring code repeat. But there is a better method: method aliasing! In Z2, a method can be an alias for another method. Their parameters must be compatible and there are a few other requirements too which I won’t describe right now. Luckily for us, parameter compatibility includes the case where a non-static method of a class Foo is an alias of a static method from another class with N + 1 parameters, where the first parameter is of class Foo. Using method aliasing, we can add only the signature of the method to classes and let the compiler forward the call to another method, with zero performance overhead. Using this, we can add the following two methods to Int:

	def Clamp(min: Int, max: Int); Intrinsic.Clamp;
	
	def Clamped(min: Int, max: Int): Int; const Intrinsic.Clamped;

Int.Clamp(Int, Int) is now an alias for Intrinsic.Clamp(ref Int, Int, Int).

Int.Clamped(Int, Int) is now an alias for Intrinsic.Clamped(const Int, Int, Int).

I shall talk more about parameters in a future post, including how ref works, but for now it is important to understand that these are just aliases. The parameters match up, are compatible, and when you call Int.Clamp, the compiler actually generates code for a call to Intrinsic.Clamp. An alias is just a formal way to say “hey, I’d like to add a new method to an interface for some purpose which leaves the heavy lifting to someone else”. The method names do not need to be identical. They are identical here because it makes sense, but the alias name can be anything.

Now it is time to see our second version of the Int class:

namespace sys.core.lang;

class Int {
	const Zero: Int = 0;
	const One: Int = 1;
	const Default: Int = Zero;

	const Min: Int = -2'147'483'648;
	const Max: Int = 2'147'483'647;

	const IsSigned = true;
	const IsInteger = true;

	const MaxDigitsLow = 9;
	const MaxDigitsHigh = 10;

	property Abs: Int {
		return this > 0 ? this : -this;
	}

	property Sqr: Int {
		return this * this;
	}

	property Sqrt: Int {
		return Int{Double{this}.Sqrt};
	}

	property Floor: Int {
		return this;
	}

	property Ceil: Int {
		return this;
	}

	property Round: Int {
		return this;
	}

	def Clamp(min: Int, max: Int); Intrinsic.Clamp;
	
	def Clamped(min: Int, max: Int): Int; const Intrinsic.Clamped;
	
#region Saturation

	this Saturated(value: Int) {
		this = value;
	}

	this Saturated(value: DWord) {
		this = value > DWord{Max} ? Max : Int{value};
	}

	this Saturated(value: Long) {
		if (value > Max)
			this = Max;
		else if (value < Min)
			this = Min;
		else
			this = Int{value};
	}

	this Saturated(value: QWord) {
		this = value > QWord{Max} ? Max : Int{value};
	}

	this Saturated(value: Double) {
		if (value > Max)
			this = Max;
		else if (value < Min)
			this = Min;
		else
			this = Int{value};
	}

#endregion
}

We can see the changes from version 1: GetMin and GetMax are gone, replaced with calls to Intrinsic when needed, Clamp is now an alias to Intinisc.Clamp and we added Clamped. Additionally, a new section has been added to the class that handles saturation. Z2 as a systems programming language is designed to have rich and performant numerical processing capabilities. Things like clamping and saturation are considered common tasks and ass such receive full support. Saturation is a lengthy section that will get repeated in multiple classes, but here we consider it not to be a problem since third party value types will generally not offer generic saturation support and us covering the basic numerical types is sufficient. This section is surrounded by the #region/#endregion tags, a purely syntactical construct that allows you to create logically related blocks in code as a tool to facilitate organizing.

And finally, this version 2 also has one additional change from what it could do in version 1, but this change is remarkable not by adding something, but by omitting something that was planned to be added but was ultimately not. Z2 supports bit rotation, not just bit shifting. This is supported with the Intrinsic class and some time ago, we had two aliases in Int for this:

	def GetRol(bits: DWord): Int; const Intrinsic.Rol32;

	def GetRor(bits: DWord): Int; const Intrinsic.Ror32;

During the design process it was decided that bit rotations are useful enough to be fully supported but not common enough to have an alias for them in Int, so these two aliases were eliminated from all core numerical types. If you need bit rotation, you can use Rol8/Rol16/Rol32/Rol64 and Ror8/Ror16/Ror32/Ror64 directly from Intrinsic.

This second version of Int, together with Double and Intrinsic have been committed to a branch in GitHub. The main branch also has some associated UT.

Next time we’ll investigate how this generic solution for clamping and other operations works with third party classes.

GitHub repository is up

The official GitHub repository is up and can be found at: https://github.com/MasterZean/z2c.

For the first phase, this repository will hold the standard library source code and some other related code, like unit testing code, documentation and benchmarks. The license is “Apache License Version 2.0”. To be honest, I have personally studied licenses in my off-time for two weeks and have reached the conclusion that not only do you need to be a lawyer to truly discern 100% of the real life implications of open-source licenses, but you also need to consult with other lawyers too. So what I’m saying is that while we do like the general principles behind open-source and we want to open-source the code, we are not married or feel strongly towards any of the individual license offerings, including Apache License Version 2.0, which may be transitory. Additionally, choosing the absolute best license at this point is beyond our means.

A few first commits were made to the repository, but for now only UT code has been added. But not the interesting UT code, but the boring kind. If you wanted to show the language to somebody by code, the UT code would be a good place to start, since it is a bunch of relatively short snippets of code, each showing off and testing some language feature sometimes in isolation, sometimes testing their flow together. In consequence the UT code can be interesting. But not the one we committed. With the amount of refactoring in PT7, things broke often and in non-obvious ways, especially when it comes to function overloading. So we added about 50 new tests, all for single parameter overloading, creating a very complete coverage of numerical types so that we can have some measure of security that overloading never breaks again. Probably about 10 tests would have been sufficient, but maybe 50 is safer.

We’ll continue to add tests to the “master” branch of the repository, but probably not 50 in one go. But it is best to add enough tests at once to reasonably cover one small feature or API element at a time.

A branch has been created with the code from “CrashCourse – 004 – Building an Int“. The next 2 posts in this series will evolve a few classes closer to their final form while explaining design decisions and once this is done, the branch will be merged into “master”. After this, the real standard library classes will start to be added to the repository, one by one, as they are documented. The documentation infrastructure still needs some work. We have documentation in the source files using comments and XML, but we would also like to evolve this tried and true formula to also work with the exact same XML tags, but externally to the source code, so that you have the choice of documenting code on the spot, with the trade-off of making the code harder to navigate, or having the documentation fully/partially in an outside file.

Now that a few pieces of code are in the repository and soon more will come, we are forced to release a super-alpha version of ZIDE. As mentioned before, the goal of ZIDE is to offer a minimum golden standard of features out of the box for editing and building software using Z2, so you don’t have to resolve to ad-hoc solutions and command line. So a version of ZIDE must exist as long as there is Z2 code, and now there is.

At the begging of April, a super-early alpha version of ZIDE will be made available, meant for developers. It won’t have many features and it will be buggy, but hopefully this early release will help us to make it better based on feedback. Unfortunately, we don’t have the time or resources for multi-platform releases right now, so this first version will only be available on Windows. Starting with the second or third release, we will have a release for Linux too.

CrashCourse – 004 – Building an Int

With PT8 development starting in the next few days, several parts of the project will get slowly released in different states of completion, the standard library source code being one of them. So it is the right time to describe a few parts of the standard library and how it evolved since its inception.

The numeric types are a good point to start, since they have a lot in common: understand one and you understand them all. As a standard library, one part of it may freely use other parts of it to accomplish some tasks. But let us suppose for a moment that all classes inside the library are independent and only serve to offer an API to clients of the library, without one referencing another one within the library. Then what is the minimal Int class?

namespace sys.core.lang;

class Int {
}

That’s it! If nobody expects Int to have a specific API, Z2 as a language does not impose any structure upon it. It is just a normal value class. But the combination of namespace plus name, the class sys.core.lang.Int is still special. It is a core class (not to be confused with sys.core, the two “core” terms have separate meanings; maybe we should fix this conflict of terms), meaning the CPU has a special understanding of it. Additionally, it is an arithmetic class. While all classes are value types, some, like Int are arithmetic implicitly, without them having an explicit API to make them behave like arithmetic types. Other third-party classes do need to have an API to conform to the arithmetic requirements. And this special treatment does not apply to other classes named Int from other namespaces.

As implicitly arithmetic, even though the Int class is empty, it still behaves as if it had several methods defined inside, like the ones commonly defined through operator overloading. All the commonly used operators in C like languages work on Int instances, like +, -, *, /, <>, ==, !=, <, , =>, ++, –, &, |, ^ and ~. They all behave as expected and you are not allowed to override them and change their meaning. Using these operators one can write complex expressions and with a few exceptions, expressions involving Ints could be copied over from C or Java into Z2.

Another thing that one does with numerical values is convert from one type to another, a task commonly done with casting. As a historical note, early versions of the Z2 design had casts, but it was found that they greatly overlapped with constructors and were eliminated. Today, Z2 has no casts and all conversions are handled though constructors. You do not cast a type to another, you construct a new instance of appropriate type, based on another instance. This is a mostly a theoretical and style based distinction, because the end result and the generated machine code are the same. As a normal class, Int has a default constructor Int{}. Conversion constructors have usually one parameter, the input value that needs to be converted. If we have a Float variable called floaty or a literal Float constant, -7.4f, we can “cast” them to Int with Int{floaty} and respectively Int{-7.4f}. And this works for all built-in numeric types, even with Bool values, like Int{true}.

As mentioned in a previous post, Z2 does not like to force you to write code that it can figure out itself or is just boiler plate code. The standard Int class could have had like 20 operators overloaded, all of them with all the parameter combinations, totaling hundreds of methods and additionally have all the conversion constructors. Instead, we choose to have this core functionality be available implicitly. Thus, the class is perfectly functional empty.

And things could be left as is. The standard library could have just a bunch of numerical classes with empty bodies, offering a few expected built-in operations. But Z2 chooses to add a bit of extra functionality to such classes. Not a huge amount, we don’t want these classes to become bloated, especially since third parties can reopen these classes and add any extra functionality they might need. Today I will show a little bit of a blast from the past, the Int class as it was a few months back. Today it is almost identical, but small changes and tweaks have been made. This simpler Int class will serve as a fine introduction on how to add value to such types and in the next posts I’ll detail how the evolution of the language has led to some changes to this class.

namespace sys.core.lang;

class Int {
	const Zero: Int = 0;
	const One: Int = 1;
	const Default: Int = Zero;

	const Min: Int = -2'147'483'648;
	const Max: Int = 2'147'483'647;

	const IsSigned = true;
	const IsInteger = true;

	const MaxDigitsLow = 9;
	const MaxDigitsHigh = 10;

	property Abs: Int {
		return this > 0 ? this : -this;
	}

	property Sqr: Int {
		return this * this;
	}

	property Sqrt: Int {
		return Int{Double{this}.Sqrt};
	}

	property Floor: Int {
		return this;
	}

	property Ceil: Int {
		return this;
	}

	property Round: Int {
		return this;
	}

	def GetMin(min: Int): Int; const {
		return this >= min ? min : this;
	}

	def GetMax(max: Int): Int; const {
		return this <= max ? max : this;
	}

	def Clamp(min: Int, max: Int): Int; const {
		if (this <= min)
			return min;
		else if (this >= max)
			return max;
		else
			return this;
	}
}

This is a rather bare bones Int class but it still offers a lot more functionality over an empty class and also serves to show our approach to library design: using this style, the difference between language features and library features is blurred. The absolute value of -7 can be obtained with -7.Abs and it looks a bit like a language feature, but the implementation is actually part of the library. Additionally, all the numeric types are extremely similar and share similar API, giving you the necessary feature parity in some situations, like when working with templates.

But let’s go slower. On lines 4-6, we have a few simple constants that do not seem that useful, giving you the 0, 1 and default values for the class. They are mostly here for feature parity with more complex numeric types, like multi-dimensional points.

On lines 8 and 9 we have two extremely important constant: Min and Max, giving us the minimum and maximum Int values. Adding these two constants to the class solves an old problem quite nicely. Where to stick these values? In C/C++, you need to include a header to access INT_MIN and INT_MAX. The recommended header changes depending on if you are using C or C++. These constants could be a #define, thus sharing the myriad of well documented problems of the pre-processor. If you are using C++ and doing things the C++ way, you need std::numeric_limits::min() and std::numeric_limits::max(). Or starting with C++ 11, besides min, there is also lowest. Why are there two? What is the difference between them? The answer is not self-evident and you need to google it to find any answer. This approach is better than using #defines, and Z2 could easily go this route, but it was decided that such a simple task should not be handled by templates. Does your type have a minimum value? If yes, just add a constant into it! You can use Int.Min to get the minimum value for Int and Foo.Max to get the maximum value for Foo if it has one. Or you can use existing instances, even literal constants, so the following samples are examples of perfectly legal expressions:

A + C * (C.Max / C.Max.Min);
A + C * (Int.Max / Int.Max.Min);
Int{Bool{Bool{Int{Bool{A}.Min.Max}}.Max}};
(true <= 6).Min <= (1 < 5).Max;

Please don’t write code like this!

On line 17 we have the Abs property defined, which returns the absolute value of the instance. On line 21 we find the very simple property that returns the square of the values. This is useful as a shorthand, when having to square some complex expression. Using Sqr, you don’t need to type it twice with a * between the two, minding side effects of the expression or having to use a temporary variable and multiplying it with itself. We find it useful and it is implemented easily inside Int, so why not have it? On line 25, we have the Sqrt property, which returns the square root of the value. This already shows interconnection of classes within the standard library: the easiest implementation of square roots on integer values is casting them to double, getting the square root and casting that result back to an integer. On lines 29, 33 and 37 we have properties that return the floor, ceiling and rounded values. For floating point values these make sense, but for integer values, they don’t really and by definition the floor of an integer is the value itself. They are included for feature parity again. As an example, you may have a template vector and run a summing lamda on it that adds together the floors of the values in the vector. This will run fine on a vector of Double as an example, but would fail to compile on a vector of Int. But because we added these feature parity APIs, the types are interchangeable and it is easier to write generic algorithms.

These methods are also logically grouped. We have one “block” doing one kind of tasks, followed by other blocks. The final block is the comparison one. Having two or more values, we often need to find the minimum and maximum of them or clamp one to a range. This is why most types in Z2, when applicable, have methods like GetMax, GetMin and Clamp. Or had, to be more precise. This is where we found that having these methods which are almost always implemented identically added to each class contradicts the principle of Z2 not making you write boiler plate code and this was changed. As explained earlier, this is how numerical types were a few months back.

Next time we’ll see how we fixed this and evolve the Int class closer to its current form.

Z2 Compiler PT7 in feature freeze

There has been a lack of new information on the blog in 2016. Sorry, I didn’t have time to write posts since I was busy with PT7. For PT7, we wanted to simplify a lot of the complexity that can be found deep within the heart of the implementation of some of the standard classes. This tuned out to be a far more lengthy task then expected. But now is the right time to iron out the last few remaining kinks, even if this means breaking compatibility.

Module system fine-tuning

First, we changed the module system. The using keyword was used to make a source file refer to another source file. After the using clause, a sequence formatted like foo.bar.(...).baz was interpreted as a reference to an existing module/source file on disk and imported as such. Like a lot of things in Z2, this is an evolution of a system from C++. In this case this was a more advanced form of f#include, but this time coupled with a powerful module system. And it worked very well. But we did find a small problem that in practice may have been a moot point, but it was still worth to fix. Top level source files were referring to lower lever modules and so on until the bottom level ones were reached. Using this hierarchy, a net semi-unavoidable increase in the number of imported symbols was noticed. The module system gave you to power to choose what to make available and what to keep private, but still a small increase was leaking from layer to layer.

So we changed the sequence after the using to refer to a fully qualified entity and decoupled it from its location on disk. I shall explain this in detail one day on the blog, but it is a variant on the C# system. But in short, a source file can refer to one or more fully qualified classes and other entities and it is the job of the compiler to supply the on disk location to them. You can still organize source codes in any way you see fit and there is no compounding public symbol pollution. And since we made sure for the standard library than fully qualified class names were in the right place on disk, this change had zero compatibility break. Compilation has gotten slightly slower because of this change, but we’ll fix this in the next versions.

Greatly simplified parameter type system

Z2 is all about combining good and best-practice inspired designs with powerful compilers to reduce the complexity and fussiness of problems. This is why we use the term of “dependency/declaration order baby-sitting”, declared it a “bad thing” and went ahead to eliminate it. Another thing we wanted to eliminate was ambiguity, especially when it came to calling functions. Like most things in life and programming, ambiguity is not binary, but a spectrum. Things that have low to medium ambiguity are often resolved in programming languages by conventions and rules. In Z2 we took this to its limit and created a language than can resolve any level of ambiguity, if it is possible to resolve of course. In consequence, the rules were extremely complicated when dealing with the types of formal parameters. We set out to create a language more powerful than C++, but with better and more sane rules, and for the most part we think we were very close to achieving this. But for formal parameters and eliminating all ambiguity by rules, we failed: we created a rule-set that is easy to learn, but almost impossible to master. The exact opposite of our goal.

We tried a solution in 2015 and now in January 2016 we tried another one. Things were better but still too complicated. So we reevaluated the problem and the value proposition of having all ambiguities resolved and came to the conclusion that… it is not worth it! We rewrote the system and now common, expected and useful ambiguities are resolved by a set of easy to learn and master rules and for the rest, we are not attempting to resolve them. We give an ambiguity related compilation error! This brings Z2 in line with other languages when it comes to the effort of learning and overall we feel that this is far better place to be in related to complexity!

Low level array clean up

Z2 has a wealth of high-level containers you should use, like Vector. It also has a couple of very low-level static vectors, RBuffer and SBuffer. Unless you are writing bindings for an existing C library, using some existing OS API or declaring static look-up tables, there is no good reason to use these low-level buffers. Still, they are in use in the heart of the standard library when calling OS features, and there was a small problem with them.

RBuffer (short for RawBuffer) is the new name starting with PT7. Before, it was called Raw. It is a raw static fixed size low-level array, like standard arrays in C. Unlike C, RBuffer has a an immutable Length and Capacity, but they are not stored in RAM. When needed, they are supplied based on compile time information. So if you have a RBuffer of Byte with a Capacity of 4, it will occupy exactly 4 bytes of RAM, not 4 plus the size of the buffer. SBuffer (short for SizeBuffer) was called previously Fixed and is low-level static fixed capacity array, that has a mutable Length that is stored in RAM and an immutable Capacity that is not stored in RAM. So a SBuffer of Byte with a Capacity of 4, will occupy RAM like this: a PtrSize sized chunk to store the Length and exactly 4 bytes. The Capacity is supplied based on compile time information, without it taking up memory. So the difference between SBuffer and RBuffer is that SBuffer has an extra field to store Length.

So far so good. The small problem of library design came from the way we were using these two types. We noticed that in most cases, a RBuffer was passed as in input, but when using RBuffer as an output, we always had an extra reference parameter to store “length”. So we refactored the standard library and now in 99% of cases a const RBuffer is used as in input and a ref SBuffer is used as an output parameter. Additionally, the low-level parts of the library no longer use pointers when those pointers are meant to be a C arrays, but use RBuffer instead. This creates a cleaner standard library.

Function overloading code refactored

All these welcome simplifications worked together and allowed us to refactor and greatly simplify the function overloading code. Simple rules give simple code and now the old super complicated implementation is gone and replaced with a far shorter one that is faster than ever!

Conclusion and future versions

This chunk of simplifications turned out very well. The language is in a far better place right now. Easier to learn, easier to master and cleaner API overall.

On the downside, implementing all this took more than expected. Starting with PT8, we want to do shorter release cycles. This means that the target final PT version moves up from about PT15 to about PT20. To compensate for this, we won’t wait until we have a very stable and feature complete compiler before we make it available for testing and instead will release a super early pre-alpha version of the compiler and ZIDE for adventurous people.

PT7 is in feature freeze and we need a couple of weeks more to fix some bugs, but starting with PT8 the standard library code will begin upload to GitHub. An account was created and I’ll write a couple of explanatory post related to the standard library and then one by one, classes will be tested, documented and uploaded.

CrashCourse – 003 – What you get for free

Happy New Year!

The winter holidays are done for now and it is time to get back to work! In December things worked out as planned. Z2 PT6 was finished, but we did not do any announcements since there is no reason to announce compiler versions which are not publicly available. PT7 development has started and it will have most of the planned features, but we are diverging a bit away from the planned features for this release. We consider Z2 to be syntactically a relatively clean language considering it aims to a have a feature set that is comparable to C++, but we did get the feedback that deep inside the implementation of some of the system classes, especially in containers and OS interaction, the language is not necessarily cluttered, but too complex instead. So we will try to address this in PT7, without breaking compatibility with the rest of language of course.

But back to CrashCourse. Last time we talked about the object model, how literal constants are still instances of classes and about constants in general. Today we shall talk about instances and what are the so-called “values”.

Z2 is a value based language, like C++. It is not a reference based language, like Java. When it comes to most languages, core numerical types are often value types and in Z2, since everything is a class and everything tries to follow one set of rules, everything is a value: all class instances are values. This does not mean that you can’t use references in Z2: you can and they behave as expected. The distinction is made by the ref keyword, which introduces references. In the absence of it, entities are values. I shall use the following short C snippet to illustrate what it means to use values, since C was at least at some point so ubiquitous:

int a = 10;
int b = a;
a = 0;

If you are ever so slightly familiar with programming, that code should be pretty self explanatory, as should “int” and the way these two variables, these two values behave. One line 1, we declare the variable “a” and assign 10 to “a”. On line 2 we declare “b” and assign to it the value of “a”. We have two separate forever independent entities here, “a” and “b”, which are stored in two different memory locations and at both memory locations you can find 10. On line 3, we assign 0 to “a”, but since “a” and “b” are independent, this does not affect “b”. This is the core principle of value types. When dealing with references, two references may refer to the same memory location and changing one variable might “change” the other too (it is not really a change, since there is just one entity accessed under two names), but this is impossible with values. In Z2 every instance of a class is a value, thus no mater how simple or complicated the class is, it behaves like “int” in the sample above.

For simple classes, this value semantic is natural and comes for free. For more complicated classes, classes that manage some resources, you need to write code in order to impose this value semantic. Without additional code, some classes, when trying to copy, might do a “shallow” copy and you can wind up in the situation of two separate instances not being logically independent. As an example, think about implementing a very simple string class that has two members: a pointer to the bytes in the string and its length. Without code to handle the copying of the string, a shallow copy will have two different string instances pointing to the same buffer. There are of course cases where you want a shallow copy, but for now we’ll consider that we want all classes to respect value semantics. Which leads us to the distinction between classes that behave like values by definition and classes where you need to write code to assure this behavior. The first case is called “plain old data” (POD for short). All core types are POD, static vectors of POD classes and classes in which all members are POD are all cases of working with POD. The primary goal of a POD class is to store data in memory. The other classes are called “manager classes”: these classes often own or manage some resources and the act of managing these resources is more important and often more complex than just storing things into memory. So the primary goal of a manager class is its side effect. If at least one member of a class is not POD, the entire class is considered not POD. But still, this distinction is mostly unimportant for now and even once we hit more advanced topics, the distinction comes down to one rule: manager classes have a destructor, a copy constructor and an assignment operator. If you add at least one of these to a class, it is automatically considered non POD and you must add all 3. But otherwise, there are no distinctions and you generally don’t care about POD or not POD. Containers like Vector care, which can do special optimizations for POD, but as a client of such containers you do not care. The introduction of POD here was probably premature, but I included it for completeness’ sake.

Now its time for some practical examples in which I will be using a POD class. To keep things simple, I won’t be using pointers inside the POD class, even though they are valid inside a POD class. Since POD values are so simple and natural, maybe the compiler can take care of a lot of things for you? Since Z2 is also a research project, we are interested in seeing how much the compiler can give you for free while still being useful and general. Values are so straight-forward, that in most cases, that what you do for copying one or verifying equality or serializing it to disk is self-evident. Why should the programmer have to write this code? How about having to write code only when the general solution is not good enough. So let’s see this class we shall be using:

class Point {
	val x = 0;
	val y = 0;
	val z = 0;
}

This is an incredibly simple 3D Point class. You should never have to write such a class in real programming situations, since the standard Z2 library comes with geometric types, but as a didactic example it will do just fine. For numerical types, we know we can get access to instances using literal constants, but how do we create a new instance of Point?

class Test {
	def @main() {
		Point{};
	}
}

By Using the “Foo{}” syntax. This creates a new instance of Foo, Point in our case. The “{}” syntax was selected to not conflict with the function calling syntax of “()”. When you see Foo{} you immediately know that is a constructor and when you see Foo() you immediately know that it is a function call. A “box” is created somewhere, probably in memory and a constructor is called using this syntax. In this case, a memory location large enough to hold a Point instance is reserved on the stack and the Point constructor is called upon it. The execution of a constructor is the only supported way of getting a new instance of a class. For numerical classes one can logically assume that each literal constant is the result of a call to a constructor, but this is just a logical abstraction. You can always call the constructor of core numerical types, so Int{} is absolutely identical to the literal constant 0, and DWord{7} is identical to 7u.

The next question: where did the constructor come from? Well, this is one of the first things the compiler offers for free: default constructors. In Z2 there is no such thing as an implicitly uninitialized variable/instance/value. Everything is initialized and every new instance is the result of a constructor. Z2 is a systems programming language, so you can explicitly have a non-initialized instance using a special syntax, but that is an advanced topic that is rarely needed in practice. So everything is initialized by a default constructor and that constructor is provided by the compiler. You can of course write your own constructor, but Z2 discourages the writing of constructors that do the default initialization logic. If the compiler provided constructor and your own do the same things, why write one? You can also write constructors that take parameters and Z2 supports named constructors. And when writing these constructors, again the compiler will help you with initialization, so these constructors should only have code that differs from the default constructor. And you can disable the default constructor for a class if you want it to be only be constructable using parameters or a named constructor.

After the execution of the Point constructor, the instance is valid and usable, so things like Point{}.x are readable. But how long is the instance valid? Until the destructor is called. The destructor is again generated by the compiler for you. The destructor will be called in most cases at the end of the statement, but the compiler might delay the execution a bit. Still it is guaranteed to be called before the end of the block. So in most cases, by the time execution hits the “;” before the end of line 3 the destructor is called. This is why I wanted to introduce the concept of POD: for POD types the destructor is guaranteed to be a “non-operation” (a NOP). Logically we still consider that the destructor was executed, but the compiler generates zero instructions for a destructor with POD types. It does nothing. Still, the instance is no longer accessible. If we want to make the instance available after the end of the statement, we need to bind it to a name using the “val” keyword:

class Test {
	def @main() {
		val p = Point{};
	}
}

This is new snippet is almost identical to the previous one: a “box” is still reserved for a new Point instance and the constructor is called. But his time, the name “p” is bound to this instance and the execution of the destructor is delayed to the end of the block. Thus we have created a local variable called “p” that can be used to read or write into our instance and is scope-bound, meaning it will be valid form the point of its declaration to the end of the block. The keyword is called “val”, not “var”, like it is encountered in many other languages, though it is functionally identical. “val” is short for “value”, contrasting with the other keyword that allows you to bind a name, “ref”, short for “reference”, which is used for references.

The same “val” keyword is used when declaring the Point class. The variables x, y and z are scope bound. Since they are inside the body of the class, the class itself is the scope. This means that the 3 variables are constructed when a Point is constructed and destructed when a Point is destructed. I mentioned before that Int{} and 0 are identical, so “val x = 0;” is identical to “val x = Int{};”. I prefer the first version since it is shorter and more natural to people coming from other programming languages.

But free constructors and destructors are not such a big deal. C++ is doing this right now! Let’s see what else we get for free looking at the full sample and its output:

class Point {
	val x = 0;
	val y = 0;
	val z = 0;
}


class FreeStuffTest {
	def @main() {
		val first = Point{};
		
		val second = Point{};
		second.x = 1;
		second.y = 10;
		second.z = 100;
		
		val third = Point{} {
			x = 1;
			y = 10;
			z = 100;
		};
		
		if (first == second)
			System.Out << "first is equal to second\n";
		else
			System.Out << "first is NOT equal to second\n";
		
		if (second != third)
			System.Out << "second is NOT equal to third\n";
		else
			System.Out << "second is equal to third\n";
		
		System.Out << "first: " << first << "\n";
		System.Out << "second: " << second << "\n";
		System.Out << "third: " << third << "\n";
	}
}

first is NOT equal to second
second is equal to third
first: 0 0 0
second: 1 10 100
third: 1 10 100

On line we declare first, a default constructed Point. All its members will be 0. On lines 11-14 we create a second variable, called “second”. Not happy with its default values, we initialize them to 1, 10 and 100, in order. Don’t worry about the multiple initializations, first by the constructor, then by the statements. The back-end compiler should take care of them. This is not a good place to use the constructor bypassing method I mentioned before. But the initialization is a bit verbose, so on lines 16-20 we initialize a third variable, called “third”, with the same values, but using a shorter syntax, available only immediately after a constructor.

Next, we get a taste of some other compiler provided features on lines 22-30: default equality checks. The compiler will automatically take care of == and != checks, using member-wise == and logical and for == and member-wise != and logical or for !=. Their purpose is to model value equality. This implementation covers most cases, and when the default is not good enough, all you need to do is provide your own implementations. If you only implement ==, you get != for free as a negation of == and the other way around. And you can implement both if you think you can write a more optimized logical expression. Default equality checks combined with standard library implementations means that you have a wide set of testable entities. Integers, strings, colors, hashmaps, hashmaps of hashmaps and so on are all testable. Other comparisons like < and > are not provided by default by the compiler, but the standard library covers this when appropriate.

Finally, on lines we 32-34 we see another big feature of the compiler capabilities: marshaling! The variables of class Point can be written to a stream without you having to implement this. This is not a case of the compiler generating a call to some toString() method and printing that string. General “toString” support is available though if needed, and yes, the compiler will generate that for you, but this is a case of the compiler generating marshaling code for Point instances. The default implementation uses a member-wise approach, marshaling each member to the stream. If this default implementation is not good enough, again, you can write your own where you can do just about anything. This marshaling solution is provided by a combination of compiler features and library support. Initially you have support for text streams and binary serialization, though the “sys.xml” and “sys.json” packages, when added to your compilation, provide automatic support for “xmlizing” or “jsonizing” most user classes. You can basically take any combination of classes and marshal them to a valid destination using statements about as complex as the lines above.

When you have some technical specification where the binary layout of serialization is a requirement, you’ll want to implement your own compliant methods. But when only wanting to get the data to disk, the default marshaling solution is deigned to be sufficient.