Breaking changes in Haxe 4.0.0

Important and breaking changes in Haxe 4

abstract Null<T>

The Null<T> type is now @:coreType abstract instead of a typedef to T. It still supports implicit conversion from and to T, as well as forwarding any field access to T, so its basic usage shouldn't be affected at all.

However, this change affects macros. Becasue it's not a typedef now, it won't be automatically followed by Context.follow and it won't be matched by the TType constructor. Instead it's now represented as a TAbstract instance, so if a macro is handling Null specifically, it needs to be changed to match TAbstract instead of TType. Also if the macro is handling TAbstract, e.g. by recursing into its underlying type, an additional match for the Null case might be needed, unless it's already handling abstracts with the @:coreType metadata.

The reason for this change is mainly to cleanup the inconsistency and confusion about Null within compiler and macro source. That is, on most static platforms, the nullable type is often different than a non-nullable type, for e.g. basic types (Int,Bool,etc), so having Null<T> represented as a mere type alias (which is what typedef is) leads to bugs and general confusion. Moreover, having an abstract Null<T> type sets the ground for possible future exploration regarding null-safety features, because with abstract types we can control implicit casting and variance easier.

Related PR: https://github.com/HaxeFoundation/haxe/pull/6380

New typed AST node: TEnumIndex

A new constructor were added to the haxe.macro.Type.TypedExprDef enum: TEnumIndex(e1:TypedExpr). It represents access to a enum index value and is generated by the pattern matcher.

Before this node was introduced, the pattern matcher was generating a (potentially inlined) Type.enumIndex call that was hard to distinguish later in the optimization filters and generators.

Related PR: https://github.com/HaxeFoundation/haxe/pull/6364

New Typed AST node: TIdent (TODO: TUnbound?)

Identifiers that are not bound to a known field or variable are now represented by a special haxe.macro.Type.TypedExprDef constructor: TIdent(s:String). These identifiers are most often introduced by the use of untyped (e.g. untyped __js__).

Before this node, such identifiers were represented by a TLocal node pointing a fake local variable with @:unbound metadata. This solution wasn't perfect, because compiler/macro developer would expect TLocal to point an actually declared variable, so having to check whether a given local variable is unbound every time is error-prone.

Related PR: https://github.com/HaxeFoundation/haxe/pull/6424

Unicode-aware lexer

Haxe now expects the source files (.hx) to be UTF-8 encoded and positions reported and expected by the compiler now contain character offsets instead of byte offsets.

This is an important change for IDE support maintainers: all display query offsets and all reported positions must be treated as character-based rather than byte-based.

This makes haxe report proper positions for source files containing multi-byte characters, such as Cyrillic, CJK and others. This also should make it easier for editors to integrate with haxe ide services, since they mostly work with character offsets as well. If the editor has the handling position conversion by e.g. encoding/decoding the file on-request, this should be disabled for Haxe 4.

Related issue: https://github.com/HaxeFoundation/haxe/issues/5163

Related PR: https://github.com/HaxeFoundation/haxe/pull/6172

1-based column numbers in compiler errors and warnings

Haxe now reports both line and column number starting from 1. This is more consistent and is what a lot of editors expect when processing build output for capturing error messages.

If you're parsing the position string (e.g. main.hx:3: characters 17-25) for further processing, please note that characters numbers are now 1-based. This can be disabled with -D old-error-format compiler argument.

Related PR: https://github.com/HaxeFoundation/haxe/pull/6391

New syntax: arrow functions

Haxe 4 now supports short arrow function syntax in the form of (a, b) -> a + b which is equivalent to function(a, b) return a + b.

While it's an obvious feature for language users, IDE maintainers will need to add support for this new syntax. In short:

// no args
() -> expr

// with args
(arg1, arg2) -> expr

// args with type hints
(arg1:Type1, arg2:Type2) -> expr

// return type hint is not supported for arrow functions due to syntax ambiguity,
// one can use type-check syntax in the function expression
(arg1:Type) -> (expr:ReturnType)

// single-arg-no-type-hint convenience special case (no parenthesis required)
arg -> expr

Related proposal: https://github.com/HaxeFoundation/haxe-evolution/blob/master/proposals/0002-arrow-functions.md