PT 9.2 Preview #1

PT 9.2 just so happens to turn out a much larger release than anticipated. It includes a major rewrite of the packaging system, making it a lot faster in hopes of the compiler getting closer to a production ready state. These packaging changes will still take some time to finish so I won’t talk about them for now, but other features will also make their way into the release and I can talk about those!

To reiterate, some of the long term goals are:

  • greatly increase Z2/C++ interoperability
  • increase compilation speed
  • do the final disambiguation and language feature cleanup rounds

With the C++ backend, Z2 can be compiled down to plain and simple C++. We generally don’t talk about or show this resulting C++, since it is an ever changing beast. The backend has multiple compilation options and if you use a well defined set of them, the compiler gives guarantees on what the C++ result will be like. But if you just randomly use the compiler with the goal of creating executables, the actual form of the resulting C++ is decided on the fly, is implementation defined and is generally tailored to be smaller and uglier for quicker compilation times. The compiler might even decide to skip all white-spaces and output the whole code on 128 (configurable) character long lines. Or it might change all you names to encoded short strings (configurable, BASE64 and other options).

This is why we talk about two separate entities. One is ad-hoc C++ code, meant to be quickly processed by a backend compiler, is implementation defined and can vary randomly between readable and obfuscated. Ad-hoc serves a single purpose: feeding a back end compiler in a final binary deliverable generation scenario. Scenarios where you don’t care about how the code looks, just what it does and you need it compiled.

The second one is interoperability code. In this mode, the resulting C++ tries to look as close as possible to both your Z2 code and the equivalent hand written code if it were originally written in C++, not Z2, but with compromises to handle the differences and needs of both languages.

So today I shall show some of the resulting interoperability mode code and to demonstrate the new features. First, let us introduce a very simple test class that has a single field called Name of type Int and a sample of its use:

class Test {
	val Name = 0;
}
val t = Test{};
t.Name = 7;
t.Name += 1;
t.Name /= 4;
System.Out << t.Name << "\n";

The snippet would of course print 2. In the past, if you had a plain class that had a simple member you wanted to have unrestricted read and write access to and there was no design document or other reason for you to expect for this member to ever be read or written to in a more complicated way, we would argue for the use of a variable over a property. If later things changed, you could then and only then change the variable to a property. And the result was always the same: OOP purists would object to the use of a public variable. They would suggest that you always use a property for public members, even if you are sure that you will never have complicated side effect based getters and setters:

class Test {
	property Name: Int {
		return name;
	}
	set (value) {
		name = value;
	}
}
private {
	val name = 0;
}

Z2 does not adopt an “one size fits all” approach to things like this and lets you decide. If you feel that you should decide on a case by case basis if each public field should be a variable or a property or instead go with a rule that all public fields should be properties, Z2 let’s you decide. Because, in the end, it might not even matter. The two versions of Test are identical. Even to the point that both the front end and back end compiler will strip away the property, leaving you just with a variable. The public API is the same for both versions, but in order to give the exact answer to what exactly happens, the answer to other questions must be known first, like if the build is debug or release mode, optimization level, inlineing, if the class is intended for dynamically linked libraries and so on.

Z2 will do instead just two things: first, give you the curtsy of keeping your property around when compiling in C++/Z2 interoperation mode, so you API looks nice and clean. Second, Z2 realizes that while there are some complex cases of getter and setters out there, in this simple case, where the property is read and write and only updates a single variable, the current syntax is too verbose, so PT 9.2 introduces this new syntax that is identical to the first:

class Test {
	property Name = name;
}
private {
	val name = 0;
}

Using this syntax, the compiler will “provide” you with getters and setters that will affect the variable that is at the right of the = sign.

Now it is time to see the resulting C++ code:

class Test {
public:
	int32 name;

	inline Test() {
		memset(this, 0, sizeof(Test));
	}

	inline Test(const Void&) {}

	inline int32 Name() const {
		return name;
	}

	inline void Name(int32 value) {
		name = value;
	}
};

The conversion is convention based. Randomly outputted C++ code from Z2 code can always be made to work with other C++ code, but the results might not be pretty. “Autogenerated” code has a reputation of being difficult to work with. So a conventions system is used to make everything look good and have predictable results. But there is also some heavy but standardized compromising always in use, because the object model, calling conventions and other details are subtly different between C++ and Z2.

But the class overall look nice and clean. I won’t discuss the details of getting this code into header files for now. Instead, let’s focus on the class. It has the same name as in Z2, but you will notice that the name variable is public. This is one of the compromises I talked about before. In C++, public/private/protected can affect you API/ABI compatibility, so by default, Z2 bypasses these possible resulting troubles by using public. One additional added benefit is that if you change a field from private to public in Z2, the C++ code does not need to be recompiled. There is an option for turning on protected and private access modifiers, but it is off by default.

Another thing you’ll notice is the second constructor (I’ll talk about the first one latter). This is another compromise. In Z2, everything is a class and there is no such thing as an implicitly unitized object. All Z2 constructors will fully initialize the instance. But C++ can skip this, mostly for built in types. The second constructor is present in every single class and is a NOP: it will leave your instance completely uninitialized. This is not for public use form C++ code, but it still must be present to satisfy Z2 API requirements. So all classes have a constructor that accepts a Void const reference and it can be always ignored because it is always guaranteed to do nothing! Nothing implies a full NOP: all members, on any depth will remain unitized, even virtual tables and other internals. Using an instance resulting from this constructor is an guaranteed error. Don’t use it!

Except for the automatic getters and setters, which use by default the “short getter/setter” naming convention (there is an option for this, default is short; with long convention, the methods are called GetFoo and SetFoo), there is the question of the first constructor. It uses memset. In the post "Class constructor performance foibles?" I detailed the problems. Cutting edge compilers, especially Clang are great at consolidating multiple small fields that are initialized with 0 and even using SSE and every trick in the book the create the fastest constructors possible. In these compilers, using memset instead of initializing has the same performance and results in the same ASM code, because memset is treated as an intrinsic. Other compilers are not that great at consolidating values, especially 8-bit ones, and will routinely be outperformed by memsetting the instance. All supported compilers have been tested and using memset is always as fast or faster than setting all fields in order. This applies the same to using initializer lists. In conclusion, using a memset is as fast as the fastest method of setting everything to zero on all supported platforms. So we have gotten around to adding this optimization to the compiler and by default you will always see memsets in constructors whenever possible. But this optimization is intentionally not aggressive. It will only be used when all the fields in the class are initialized with 0 bits. Otherwise, it will initialize them as usual. It handles pointers, like we can see with String:

String::String() {
	this->data = nullptr;
	this->length = 0;
	this->capacity = 0;
}

becomes…

String::String() {
	memset(this, 0, sizeof(String));
}

It is also aware of types it has already optimized, so even if you have a non POD class that is embedded in another class, the whole deal can be optimized and flattened down to a single memset, instead of the host class calling the child class’ constructor, which memsets and the doing a separate memset for the rest of the members:

SystemCPU::SystemCPU(): Vendor(_Void) {
	new (&this->Vendor) ::String();
	this->MMX = false;
	this->SSE = false;
	this->SSE2 = false;
	this->SSE3 = false;
	this->SSSE3 = false;
	this->SSE41 = false;
	this->SSE42 = false;
}

becomes…

SystemCPU::SystemCPU(): Vendor(_Void) {
	memset(this, 0, sizeof(SystemCPU));
}

And finally, it also handles classes with virtual methods correctly:

Stream::Stream() {
	memset(&this->pos, 0, sizeof(Stream) - __Z_MEMBER_OFFSET);
}

For classes with virtual mebers, the memset starts at the offset of the first field, making sure to not nuke the vtable. The actual memset that is generated varies based on backend compiler and class layout, so don’t take the actual code as set in stone, only what it does: if a constructor logically ends up writing only 0 bits into the entire instance, barring vtables and what-not, an appropriate memset optimization will kick in and this will guarantee that constructors on really old compilers are more competitive with the latest Clang.

Next, let us look at one of the samples form the org.z2legacy.ut package, in the access folder:

namespace org.z2legacy.ut.access;

using org.z2legacy.ut.access.Foo;

class FailPrivate01 {
	def @main() {
		val p = Foo{};
	}
}

The contents of the sample are not important here: it just tests that the private constructor of Foo is indeed not accessible. The new minor feature is that you can now write:

namespace org.z2legacy.ut.access;

using Foo;

class FailPrivate01 {
	def @main() {
		val p = Foo{};
	}
}

When the using statement is followed by an unqualified class name, it will always assume that it is in the same namespace as the one specified in the namespace statement, so using org.z2legacy.ut.access.Foo means the same as using Foo. This may lead you to the question: how do you handle classes that are not within a namespace? The short answer is: you can’t.

But fret not! We removed the ability to have classes outside of namespaces! Not for the above mentioned reason, but because try as we might, as soon as packages started to grow, public namespace pollution became more an more of an issue. If anybody can add names to the public namespace, it is only a question of time before two different packages will define two different classes with the same name. With mandatory namespaces, this issue is greatly lessened.

So starting with Z2 PT 9.2, all your classes must be added to a namespace. This is the first breaking change in the language, but the language is very young, so it should not be a problem. It was either adding a breaking change, or realizing years from now, when it is too late, that namespace pollution is indeed a severe problem affecting actual code.

I’ll go in and change the content on the site to reflect this change.

As a bit of an tangentially related curiosity, Z2 does away with declaration orders being meaningful when they are not needed, as stated before. This is great for classes and methods, where one class can have access to another class that is defined in the same file, only latter. You no longer have to babysit orders and you only care about public or private access rights. But this also affects the namespace statement, so the above example is 100% identical to the case where the namespace statement is not the first line in the file, but placed somewhere more awkward, like:

using Foo;

class FailPrivate01 {
	def @main() {
		val p = Foo{};
	}
}

// DO NO LOOK HERE!!!!!!! I AM HIDING!!!!
namespace org.z2legacy.ut.access;

You can have only one namespace statement per source file, so all the classes in a file must be in the same namespace. It makes sense for it to be in the beginning of the file, maybe even the first statement, since it affects the whole file, but you can place it anywhere outside of a class definition.

The error reporting within the compiler has been upgraded. A new component was introduced to centrally handle all error reporting. This will also allow for internationalization of error messages and the assignment of unique error codes, but the list is not agreed upon yet. Additionally, some error messages have been improved, as one can see in this command line screenshot:

error

To review the contents of this preview, I talked about:

  • new shorthand syntax for properties
  • memset optimization for zeroing constructors
  • new shorthand syntax for the suing statement
  • new error reporting component

At least one more preview and maybe even a minor release will be created before PT 9.2 is released.

Advertisements