Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nullable References #355

Open
jdmichel opened this issue Nov 14, 2018 · 94 comments
Open

Nullable References #355

jdmichel opened this issue Nov 14, 2018 · 94 comments

Comments

@jdmichel
Copy link

Is there already a plan for supporting the C# 8 features related to nullable references?

What might this look like in VB?

void Foo(Bar a, Bar? b) {} ->
Sub Foo(a As Bar, b As Bar?)
or
Sub Foo(a As Bar, Optional b As Bar)
or even something else?

Even if you decided on the (breaking change?) Optional keyword then I suppose the IDE could support automatic conversion of the first syntax to save keystrokes.

There are bazillion details and corner cases, but I didn't see an existing issue on this topic, and I'm curious.

@reduckted
Copy link
Contributor

Sub Foo(a As Bar, b As Bar?)

That would make the most sense, and aligns with the syntax for Nullable(Of T).

@jdmichel
Copy link
Author

I don't think that we need nearly the complicated implementation that the C# team chose. I would instead just trust the type annotation. If it says it's nullable, then just assume it might be null.
Introducing Nullable Reference Types in C#

sub M(ns as string?) ' ns is nullable
	WriteLine(ns.Length) ' Warning A: null reference
	if ns isnot nothing then
		WriteLine(ns.Length) ' Warning A: null reference
		dim s as string = ns ' Warning B: nullable to non-nullable
		WriteLine(s.Length) ' ok, not null here
		
		dim s2 = if(ns, "") ' If operator smart enough to infer non-nullable?
		WriteLine(s2.Length) ' ok, not null here

		if ns is nothing then
			return ' Unlike c#, this didn't buy us anything. ns is still nullable
		end if
		WriteLine(ns.Length) ' Warning A: null reference
	end if 
end sub
 
sub N(s as string) 
	if s is nothing then ' Warning C: 's' not nullable
	end if
	
	s?.Foo() ' Warning C: 's' not nullable
	
	s.Foo() 
	
	s = nothing ' Warning B: nullable to non-nullable
end sub

sub Test() 
	dim ns as string? = "Test"
	dim s as string = "Test2"
	M(ns) ' nullable to nullable is no prob
	M(s) ' Warning D: non-nullable to nullable. 
	N(ns) ' Warning B: nullable to non-nullable
	N(s) ' Fine
	N(if(ns, "")) ' If operator smart enough to infer non-nullable?
end sub

class Person 
	public FirstName as string
	public MiddleName as string?
	public LastName as string
end class

structure PersonHandle
	public person as Person' Warning E
end structure

sub Test2(p as Person)
	p.FirstName = nothing ' Warning B
	p.LastName = p.MiddleName ' Warning B
	dim s as string = nothing ' Warning B
	dim a(9) as string ' Warning F
end sub

I think it would be nice to make the warnings above separate so that the project can choose to ignore some of them.
Warning E and F above were called out as special cases that C# chose to ignore. In both cases there is no way for the compiler to infer a default value for the non-nullable type.

I think this gives us the easiest possible path to this feature, while retaining 99% of the benefit of preventing most null reference exceptions.

@KathleenDollard
Copy link
Contributor

Here are my current thoughts on this...

C# nullable reference types is a wild experiment. It's an attempt to make people rethink the way they work with and think about every single usage of reference types. Yes, there is significant compiler support. But that compiler support is to create a bazillion warnings.

It is not just the declared type. If you look at the fourth line of @jdmichel code, a warning is clearly not needed. Humans and the flow control graph (CFG) both know this cannot be null. You would not want flow control ignored, because then the code you write today to correctly manage nulls would not work. With flow control, an null guard (assert) will prove that the value cannot be null, regardless of what was known before the null guard. In the method M in my quick review, nothing should throw a warning except the first line. Otherwise, it's already protected code. IOW, you would get so many unimportant warnings in good code that the important ones would get lost.

As far as VB, we need to first see this grand experiment get traction in C#. Then we need to look at the question more deeply as to whether this makes sense in VB.NET. A feature whose purpose it to break code and force you to make significant changes to use it does not feel very VB-like - and that is thinking only of the Option Strict/Option Explicit approach. It really seems to make no sense outside that.

Someone I respect was quite surprised that I am hesitant to think this feature belongs in VB. Then they sat through a single design meeting (there have been many, many) and looked at me and said "OK, I get it"

I can't say I'm at a final decision on this. But at the current point I think the major special thing about VB is that the code does what it says it is doing - you can see and quickly comprehend that. Part of that is syntax, and maybe we could find good syntax. But part is also decades of experience that reference types are nullable and that string is nullable. Making string not null in code that has a particular gesture somewhere else in the code base, and still nullable where this gesture doesn't exist - that feels chaotic to me.

@zspitz
Copy link

zspitz commented Jan 6, 2019

@jdmichel

To add to @KathleenDollard 's comment (It's not just the declared type...), if we don't allow flow control to affect the understood type, the only way to use this without a warning would be to declare a new variable (Dim s As String = ns), which would require at a minimum some kind of type cast (Dim s = CType(ns, String)) in order to avoid a warning. This is not a good solution -- it would put nullable reference types in the same situation as type-checking today:

Dim o As Object
' fill o
If TypeOf o Is Random Then
    ' we cannot treat o as an instance of Random
    Dim rnd As Random = o
    ' we have to go through rnd
    ' alternatively, we could use CType
End If

Not having to do this is the "killer app" of pattern matching for VB.NET -- even newcomers to pattern matching immediately "get" this benefit. There is also a long-standing suggestion to rely on flow control within type-check blocks (#172), in order to avoid this.

@jdmichel
Copy link
Author

jdmichel commented Jan 6, 2019

First, thank you for responding to my suggestion.

My perspective comes from ~15 years spent writing C++ in a style that prefers use of references over pointers in 99% of all code. So I'm used to thinking of "reference" as something that cannot be null, and it always felt like Java (then C# and VB.NET) re-approriated the term reference for a concept that feels more like a pointer.

In any case, one of the great benefits of what we called "modern C++" back in the 90's was that this use of references made it easier to reason about code, because you were free to assume that such code could never contain null pointers. It decreased the cognitive load of all of our code, and meant that in practice the only time we had to deal with null was when interfacing with older C-style code from third party libraries. We wrote simplified wrappers around such code, and this is the only place you would see "if (foo_opt != null) { Foo& foo = *foo; ...}" guards to translate between the two styles. I think the C# team is going to already handle annotating most dotnet libraries with nullable type information, so I would expect these types of checks to be mostly unnecessary in VB/C# for teams that choose to embrace defaulting to non-null. Therefore, I expect it to be very rare to ever need flow control analysis for nullability. Which is why I thought VB could take a chance on embracing the non-nullable option (Option Explicit Nulls?). Although it might be nice to support flow control analysis to infer non-nullable (shadow variables?), my point is that you get 90% of the benefit without it, because in practice your code is not going to be asserting or checking for null at all, because within the projects that embrace this style then all the variables/parameters are already going to be non-nullable, and it will be extremely rare for any code to make use of nullable reference types at all. For existing code, or for people who don't want to embrace this new style, then everything continues to work as it does now.

Incidentally, I also don't understand #172, because I don't understand why I would ever have a variable/parameter of type Object that I then check for the type. In my style of VB coding (Option Explict, Option Strict) you would just rarely/never find yourself in this situation in the first place. Almost all my code would have strong types, and it isn't much trouble to wrap any code that is not type safe.

For the places that do pass nullable, I think it could be nice to make it easy to cast away nullability.

sub DoSomething(foo as Foo?) 
    if foo is nothing then
        ' handle the case where it's null
        ...
    else
       dim f = foo! ' Use exclamation to cast away the nullability to avoid the need for flow control
       ... ' Use f instead of foo from here on out because any use of foo. is now a warning (or even an error)
    end if
end sub

The same goes for dynamic types:

sub DoStuff(obj as Object) 
    if typeof obj is Foo then
       dim foo = directcast(obj, Foo)
       ' Use foo instead of obj
    else if typeof obj is Bar then
    else
    end if
end sub

That being said, thinking back, I don't think either of the above code was used much at all, because we would just never write code that took a Foo? or Object in the first place. That sort of thing might only exist at the edges where we called other peoples code, and those also usually required extreme testing for things like throwing unexpected exceptions or otherwise behaving in undocumented or incorrect ways. Or they would have complex interfaces to handle flexibliity that we didn't want or need, so we would wrap them with our own simpler type-safe non-nullable types.

Anyway, the point of all of this is to hopefully eliminate what is in my experience the most common class of bugs in Java/dotnet code, and even more importantly to make code easier to reason about by eliminating the need to consider null in most of our code. I spent 8 years working on a large Java program, and NullReferenceExceptions were extremely common, and I think embracing non-nullable defaults style would have prevented most of those bugs. It seems like the c# team was finding the same thing as they refactored existing dotnet libraries in this style. I just think they wasted alot of effort by supporting flow analysis.

@zspitz
Copy link

zspitz commented Jan 6, 2019

@KathleenDollard

the major special thing about VB is that the code does what it says it is doing - you can see and quickly comprehend that.

Today, when I define a variable of String in my code, I am actually saying "this should be a String, in which case I have all the behaviors of a String; but it might be this other thing which in no way behaves like a String, and any attempt to use it like a String is liable to fail". That seems a rather large underlying concept that's not immediately obvious from the code.

As you've noted, "decades of experience that reference types are nullable and that string is nullable" have firmly entrenched this concept into the heads of every .NET developer from day one; but it's certainly not something obvious from a casual reading of the code.

@pricerc
Copy link

pricerc commented Jan 6, 2019

String is annoying. It's a reference type that likes to pretend it's a value type (e.g. a reference type shouldn't need to be immutable). And then there's dealing with whether Empty needs to be treated the same as Nothing. And then in VB, we allow people to pretend they're not even strings and do things like math(s) on them. I'm in favour of e.g. removing + as a concatenation operator.

Back on topic: I do, however, kind of like the idea of decorating method parameters so that code analysis could know whether Nothing is a reasonable value to pass in, and post warnings about possible causes for concern.

In the discussion on the linked article, a comment is made by Mads that guard checks for null are probably still a good idea for public APIs, since external code still pass a null in and so you'd still need to be able to deal with them.

@KathleenDollard
Copy link
Contributor

KathleenDollard commented Jan 7, 2019

@jdmichel I think if we were designing the behavior for new language we would not have null as a default.

It is an interesting opinion that we rely only on the declared nullability and not flow in Visual Basic.

I still have concern about changing. the meaning of code - the foundational change that "Option Null Strict" or similar would bring to code.

However, you've added a layer to my thinking which is to not change the default, but allow declaration of a non-null, including in parameters. Programmers would have to do more work to track values through, including the examples you gave above. Thus there would have to be "cast-like" thing as well as a declaration site gesture. One (knee-jerk, not well thought out) idea is something that looked like pattern matching (and may or may not actually be).

As an example: this code from here bothers me. I don't yet see how you accomplish this without full safety and no warnings without expanding your suggestion. Here's a copy of the part that troubles me:

if ns isnot nothing then
		WriteLine(ns.Length) ' Warning A: null reference
		dim s as string = ns ' Warning B: nullable to non-nullable
		WriteLine(s.Length) ' ok, not null here

As a possible solution, I'll rewrite temporarily using ! as the not-null specifier and a version of pattern matching syntax (which we are certainly not decided on).

Dim ns As String
WriteLine(ns.Length) ' No warning, this is oblivious as today
If ns Is String! Into s
      WriteLine(s.Length) ' No warning, it's for real non-null
End If

' For intentional nullable
Dim nn As String?
WriteLine(nn.Length) ' Warning, it can definitely contain null

In that direction, Option Null Specified (or similar) would simply outlaw the first line of my sample.

Code would always do the same thing.

C# did not go the route of specifying everything, because they felt it would be too much explicitness - but VB thrives on explicitness. I actually think if we did this I'd like to fall even deeper into the explicitness to:

Dim ns As String
WriteLine(ns.Length) ' No warning, this is oblivious as today
If ns Is NonNull String Into s
      WriteLine(s.Length) ' No warning, it's for real non-null
End If

' For intentional nullable
Dim nn As Nullable String

Anyway, these are just my top of the head thoughts on this.

@pricerc
Copy link

pricerc commented Jan 8, 2019

Slightly random thought

While I get where they're coming from, the Nullable Reference type being introduced in C# is not quite the same as a Nullable Value type, because Nullable Value types are built on Nullable(Of T).

To be consistent, Nullable(Of T) could be extended to allow T to be a Class or a Structure. Then the 'Value' property may be Nothing or an instance of T, and for a Class, HasValue just returns Value IsNot Nothing

Then

Public Sub GetCask(value as Wine)
	If Wine Is Nothing Then Throw New ArgumentNullException
	Console.WriteLine($"{value.Name} {value.Vintage}")
End Sub

could become

Public Sub GetCask(value as Wine?)
	If Not Wine.HasValue Then Throw New ArgumentNullException
	Console.WriteLine($"{value.Name} {value.Vintage}")
End Sub

And at the same time decorating the method with the intention to allow nulls.

@zspitz
Copy link

zspitz commented Jan 8, 2019

@KathleenDollard

if we did this I'd like to fall even deeper into the explicitness

Would such a syntax (Nullable / NonNullable) be extended to value types, leaving two ways to specify nullable value types?

Is value type? Intentionally can contain Nothing Cannot contain Nothing Ambiguous about Nothing
Value type Integer? Integer
Value type Nullable Integer Integer
(NonNull Integer would be invalid)
Reference type Nullable String NonNull String String

Or would there be two separate syntaxes for value types vs reference types:

Is value type? Intentionally can contain Nothing Cannot contain Nothing Ambiguous about Nothing
Value type Integer? Integer
Reference type Nullable String NonNull String String

Either possibility would add cognitive load to code authors.

And if we're stuck with using symbols for explicitly-nullable- and non-nullable-reference-types, then the VB.NET explicitness argument falls down. VB.NET is easy to read and understand because the only symbols used in the language are almost universally understood even from a non-programming perspective -- e.g. mathematical operators (+, -, *, /), comparison operators (<, >, =) or order of operations ((,)). Virtually everything else is represented with keywords. But I think a design decision that would result in more symbols runs counter to readability -- APL with its' multitude of symbols may be explicit, but is less readable, because you have to know what the symbols mean.

I still have concern about changing. the meaning of code - the foundational change that "Option Null Strict" or similar would bring to code.

Having used Typescript both before and after undefined and null were isolated into their own types, I can attest that it took me about two weeks to make the mental transition between "the string type obviously includes undefined or null as one of its possible values" to "why in the world should the string type treat undefined / null as a possible value?"

I would suggest that it's not such a foundational change. Even today, Nothing has little value when typed by the compiler as String -- you can't read any properties of a normal String, call any of it's methods; if you try to pass it into another method, you're liable to get an ArgumentNullException. Virtually the only thing you can do, is filter out the possibility that the String might be Nothing before attempting to use it as a regular String.

@jdmichel
Copy link
Author

jdmichel commented Jan 8, 2019

I think this is a great discussion, and I hope we're closing in on a solution that everyone will like.
I have several thoughts after re-reading and thinking about the above.

  1. I think it's completely in the character of VB to have an option that changes the meaning and behavior of existing code. Option Strict, Compare, and maybe even Explicit all do this. So it still seems to me that something like Option NonNull On that changed the default for all reference types to prevent null would be easy to understand. I even think this could make sense as errors instead of warnings, because you are still opting in to the behavior. I wish this were easy to hack in to VB so that I could try to refactor my largest VB project to try it out. I've been mostly thinking of it from the perspective of something I'd only use for new code, but maybe it's more realistic to think of it from the perspective of VB teams deciding to adopt it for existing code. My gut says it would be similar, but easier, than enabling Option Strict On and Option Explicit On for an existing project.

  2. I haven't been following proposals for pattern matching. (Is there a better name for this? I find it confusing.) It looks like a replacement for TryCast to me, so maybe it would be less confusing as:

if trycast ns as string into s then
    WriteLine(s.Length)
end if

(Btw, a few years ago I started using Anthony Green's April Fools font to let me try VB with all lowercase keywords, and I've grown to greatly prefer that look. I always typed VB keywords as lowercase then relied on the IDE to fix the case, but I find it's enough feedback for the IDE to colorize keywords so that I know it understood me.)

Maybe we could even support a default for the 'as' clause to cast away the nullability so that the following would be a more concise way to do the same thing:

if trycast ns into s then
end if

Or maybe it's more consistent if you rearrange the first syntax to:

if trycast ns into s as string then...

That way it's clear that the 'as string' part can be inferred if Option Infer On is set.

  1. I like keywords over special symbols, but have to admit that in one project that used nullable value types I found it much clearer to just use question marks. But maybe we don't actually need the exclamation point at all if we have something like pattern matching (still don't like that term) above?

  2. I think Option NonNull(able?) On makes reference types more consistent with value types, and therefore easier to understand. To ensure symmetry you would also want to support the pattern matching syntax for value types.

dim apples as Nullable(of Integer) = 42
if trycast apples into count as integer then
    SendApples(count)
end if

This seems more readable for both value and reference types, but with Option NonNull Off you could still support the following without any new keyword (just an implicit new name for the new kind of non-nullable reference):

dim ns as Nullable(of string) 
if trycast ns into s as NonNullable(of string) then
    Console.WriteLine(s.Length)
end if

Turning on Option NonNull(able) would then simply bring the defaults for references in line with the defaults for value types and let you simplify the above syntax to:

dim ns as string?
if trycast ns into s as string then
    Console.WriteLine(s.Length)
end if

This feels very much in the spirit of VB to me, but maybe I'm just too close to it, and it's been a pet peeve of mine since I first used VB (and others like Java, C#, Python, JavaScript), because I was already used to C++ non-nullable references.

@zspitz
Copy link

zspitz commented Jan 8, 2019

@jdmichel

I find it (pattern matching) confusing.

My mental model of pattern matching looks something like this:
image
You define a pattern, or set of patterns, and the object in question is tested against each pattern if it matches. Pattern syntax could theoretically describe a specific type:

Dim o As Object
Select Case o
    Case String: Console.WriteLine("It's a string")
    Case Integer: Console.WriteLine("It's an integer")
End Select

or something else:

Dim o As Object
Select Case o
    Case 5 To 10: Console.WriteLine("It's a number or numeric string between 5 and 10")
    Case Like "A*B": Console.WriteLine("It's a string that starts with A and ends with B")
End Select

An additional goal of pattern matching is to extract all or part of the object into new variables:

Dim o As Object
Select Case o
    Case String Into s:  Console.WriteLine($"Length of string: {s.Length}")
    Case Integer Into i: Console.WriteLine($"i ^ 2 = {i^2}")
    Case With {.LastName = "Smith", .DOB Matches > #1/1/1985# Into Since1985}
        Console.WriteLine("Much more concise than the following alternative, with only a single new variable")
        Console.WriteLine("Also works if the object is not of the known type Person")
        ' If TypeOf o Is Person Then
        '     Dim p As Person = o
        '     If p.LastName = "Smith" AndAlso p.DOB > #1/1/1985# Then
        '         Dim Since1985 = p.DOB
        '     End If
        ' End If
    Case (String Into s, Integer Into i)
        Console.WriteLine($"Tuple of string '{s}' and integer '{i}'")
End Select

It's rather more than just a replacement for the TryCast-into-a-variable idiom, because patterns could theoretically represent much more than just a type + variable assignment.

Because pattern matching is a generalized syntax for matching patterns, and not just typechecking+variable assignment, I don't think it appropriate to modify the pattern matching syntax just for this use case.

@pricerc
Copy link

pricerc commented Jan 8, 2019

Although the discussion is nominally about nullable reference types; Is that not really a misnomer, since reference types are by definition nullable?

It looks to me more that we're talking about the potential of 'Not Nullable' reference types as additional tool in our toolbelt.

What we're really talking about is a new semantic option (compiler flag) that allows the compiler and/or code analysis to make better predictions about potential problems (primarily) with passing reference-type arguments to Public(and maybe Friend) methods (including property setters) that don't really want null values.

So. The way I see it, what I'd like from this concept is:

  1. a mechanism where the compiler generates appropriate null guards for me.
  2. the compiler warns me if I'm passing Nothing to a Nothing-not-wanted parameter.

From the C# blog:

Naively, this suggests that we add two new kinds of reference types: “safely nonnullable” reference types
(maybe written string!) and “safely nullable” reference types (maybe written string?) in addition to the
current, unhappy reference types.

We’re not going to do that....

I don't think that logic applies to VB. I think it would be preferable to give people the option of using a new way of doing things, if it adds value to what they're doing, using exactly the constructs they're describing.

So I think adding both ! and ? decorations to reference type method parameters has merit to permit per-method, explicit declaration of intent. When introduced along with an appropriate Option.

The name of the option should describe clearly the handling of null references:

  1. the 'Legacy' way, references are implicitly nullable
  2. the 'New' way, references must be explicitly marked as nullable

So how about something like Option NullableReference Implicit|Explicit ? where Implicit is the current behaviour, and Explicit inverts it. For short form, maybe NullRef.

When combined with ! and ?, you can mix & match:

Option NullRef Implicit

Class Wine
	Public Property Name As String
	Public Property Vintage As Integer
End Class

' 1) NullRef Implicit; this enforces NotNullable semantics on `value`
Public Sub GetCaskExplicitNotNull(value as Wine!) 
	' 1a) Compiler inserts null reference check here, 
	'		throws ArgumentNullException(NameOf(value)) if value is Nothing
	' 1b) value is defined, all good
	Console.WriteLine($"{value.Name} {value.Vintage}")
End Sub

' 2) NullRef Implicit; this works the same way it always has.
Public Pub GetCaskImplicitNullA(value as Wine) 
	' 2a) Compiler does *not* insert null reference check
	' 2b) Seeing no user-defined reference check, compiler issues warning.
	' 2c) Potential RunTime NullReferenceException:
	Console.WriteLine($"{value.Name} {value.Vintage}")
End Sub

' 3) NullRef Implicit; this works the same way it always has.
Public Pub GetCaskImplicitNullB(value as Wine) 
	' 3a) Compiler does *not* insert null reference check
	' 3b) Seeing user-defined reference check, compiler issues no warning.
	If value Is Nothing Then Throw New ArgumentNullException(NameOf(value))
	' 3c) value is defined, all good:
	Console.WriteLine($"{value.Name} {value.Vintage}")
End Sub

' 4) NullRef Implicit; this is redundant, but still legal.
Public Sub GetCaskExplicitNullA(value as Wine?)                                           
	' 4a) Compiler does *not* insert null reference check, 
	' 4b) Seeing no user-defined reference check, compiler issues warning.
	' 4c) RunTime NullReferenceException if value is null:
	Console.WriteLine($"{value.Name} {value.Vintage}")                        
End Sub

' 5) NullRef Implicit; this is redundant, but still legal.
Public Sub GetCaskExplicitNullB(value as Wine?)                                           
	' 5a) Compiler does *not* insert null reference check.
	' 5b) Seeing user-defined reference check, compiler issues no warning.
	If value Is Nothing Then Throw New ArgumentNullException(NameOf(value))
	' 5c) value is defined, all good:
	Console.WriteLine($"{value.Name} {value.Vintage}")                       
End Sub

' 6) If Nullable(Of T) extended to allow T to be a reference type, then:
' NullRef Implicit; this is redundant, but still legal.
Public Sub GetCaskExplicitNullC(value as Wine?)                                           
	' 6a) Compiler does *not* insert null reference check
	' 6b) Nullable(Of T) lets you do this; and seeing a user-defined reference check, compiler issues no warning.
	If Not value.HasValue Then Throw New ArgumentNullException(NameOf(value))
	' 6c) value is defined, all good:
	Console.WriteLine($"{value.Name} {value.Vintage}")                       
End Sub

Public Sub CallGetCasks
	Dim newWine As Wine = Nothing
	
	' Compiler warning, RunTime ArgumentNullException from compiler-added guard code
	GetCaskExplicitNotNull(newWine) 
	
	' No compiler warning, RunTime NullReferenceException (from called method)
	GetCaskImplicitNullA(newWine) 
	
	' No compiler warning, RunTime ArgumentNullException (from called method)
	GetCaskImplicitNullB(newWine) 
	
	' No compiler warning, RunTime NullReferenceException (from called method)
	GetCaskExplicitNullA(newWine) 
	
	' No compiler warning, RunTime ArgumentNullException (from called method)
	GetCaskExplicitNullB(newWine) 
	
	' No compiler warning, RunTime ArgumentNullException (from called method)
	GetCaskExplicitNullC(newWine) 
End Sub

or

Option NullRef Explicit

Class Wine
	Public Property Name As String
	Public Property Vintage As Integer
End Class

' 1) NullRef Explicit; this is redundant, but still legal
Public Sub GetCaskExplicitNotNull(value as Wine!) 
	' 1a) Compiler inserts null reference check here,
	'		throws ArgumentNullException(NameOf(value))  if value is Nothing
	' 1b) value is defined, all good
	Console.WriteLine($"{value.Name} {value.Vintage}")
End Sub


' 2) NullRef Explicit; value becomes 'NotNullable' 
Public Pub GetCaskImplicitNotNullA(value as Wine) 
	' 2a) Compiler inserts null reference check here,
	'		throws ArgumentNullException(NameOf(value))  if value is Nothing
	' 2b) value is defined, all good
	Console.WriteLine($"{value.Name} {value.Vintage}")
End Sub

' 3) NullRef Explicit; value becomes 'NotNullable' 
Public Pub GetCaskImplicitNotNullB(value as Wine) 
	' 3a) Compiler inserts null reference check here,
	'		throws ArgumentNullException(NameOf(value))  if value is Nothing
	' 3b) Compiler sees you have too, issues warning.
	If value Is Nothing Then Throw New ArgumentNullException(NameOf(value)) 
	' 3c) value is defined, all good
	Console.WriteLine($"{value.Name} {value.Vintage}")
End Sub

' 4) NullRef Explicit; value is Nullable
Public Sub GetCaskExplicitNullA(value as Wine?)                                           
	' 4a) Compiler does *not* insert null reference check, 
	' 4b) Seeing no user-defined reference check, compiler issues warning.
	' 4c) RunTime NullRefException if value is null:
	Console.WriteLine($"{value.Name} {value.Vintage}")                        
End Sub

' 5) NullRef Explicit; value is Nullable.
Public Sub GetCaskExplicitNullB(value as Wine?)                                           
	' 5a) Compiler does *not* insert null reference check.
	' 5b) Seeing user-defined reference check, compiler issues no warning.
	If value Is Nothing Then Throw New ArgumentNullException(NameOf(value))
	' 5c) value is defined, all good:
	Console.WriteLine($"{value.Name} {value.Vintage}")                       
End Sub

' 6) If Nullable(Of T) extended to allow T to be a reference type, then:
' NullRef Explicit; value is Nullable
Public Sub GetCaskExplicitNullC(value as Wine?)                                           
	' 6a) Compiler does *not* insert null reference check
	' 6b) Nullable(Of T) lets you do this; and seeing a user-defined reference check, compiler issues no warning.
	If Not value.HasValue Then Throw New ArgumentNullException(NameOf(value))
	' 6c) value is defined, all good:
	Console.WriteLine($"{value.Name} {value.Vintage}")                       
End Sub

Public Sub CallGetCasks
	Dim newWine As Wine = Nothing
	' Compiler warning, RunTime ArgumentNullException from compiler-added guard code
	GetCaskExplicitNotNull(newWine) 

	' Compiler warning, RunTime ArgumentNullException from compiler-added guard code
	GetCaskImplicitNotNullA(newWine) 

	' Compiler warning, RunTime ArgumentNullException from compiler-added guard code
	GetCaskImplicitNotNullB(newWine) 

	' No compiler warning, RunTime NullRefException (from called method)
	GetCaskExplicitNullA(newWine) 

	' No compiler warning, RunTime ArgumentNullException (from called method)
	GetCaskExplicitNullB(newWine) 

	' No compiler warning, RunTime ArgumentNullException (from called method)
	GetCaskExplicitNullC(newWine) 

End Sub

Assignments between nullable and not-nullable references should have the same semantics as assignments between T and Nullable(Of T) has today.

@zspitz
Copy link

zspitz commented Jan 9, 2019

@pricerc

Although the discussion is nominally about nullable reference types; Is that not really a misnomer, since reference types are by definition nullable?

Not quite, although it is a little confusing. The C# proposal actually consists of two basic things:

  1. Reinterpret every current usage of a reference type as excluding Nothing, so warnings can be issued on potential misuse; in other words, current reference types -> non-nullable reference types.
  2. A syntax variation is now needed to define reference types that can include Nothing; this is the syntax that is being added to the language -- a syntax for nullable reference types. Hence, the name.

I think it would be preferable to give people the option of using a new way of doing things, if it adds value to what they're doing

I would suggest that we first have to consider what the right design should have been. In VB.NET, the shape of an object as recognized by the language comes from its type, wherein are defined the methods, properties, events, constructors etc. that are relevant for objects of a given type (inheriting or implementing). But Nothing has a completely different shape -- none of the shape of the type applies to Nothing. Therefore, bundling Nothing as a valid value for reference types is a sort of violation of the contract implied by the type's shape.

It's true that the CLR type system does treat Nothing as a value compatible with reference types, but what's being proposed here is to add a refinement to the virtual type system that already exists on top of the CLR type system; akin to how Typescript is a virtual type system on top of Javascript, which has no type system in the language at all. (This also has the benefit of not being a change to the CLR, or even to IL emitted by the compiler.)

If we can agree that ideally Nothing should be excluded from the domain of a given reference type, then saddling every future usage of reference types with an explicit annotation to denote non-nullability seems inappropriate.

The only difference here between C# and VB.NET, is that C# has a large community of enthusiasts, who would be happy to accept additional warnings as the price of more correct code, and are thus willing to drive these kinds of changes; my perception is that the VB.NET community is smaller, and less users who would be willing to pay this price.

@Bill-McC
Copy link

Bill-McC commented Jan 9, 2019

I honestly don't get this. Every time I think of the edge cases I keep coming up with a cat in a box ....

At what point during instantiation is an objects fields null or not null. If they cannot be null, then all instantiation has to be to a default which is just another form null with another a value. VB already went down that road with strings and equality testing with Nothing.
And the complexities of ensuring there is no inadvertent assignment of Nothing to a reference type that has to have a default value becomes daunting when considering query language.
We've added constructs to deal with nested nulls, but that defaults to null anyway so would be pointless if mulls were no longer allowed.
In the end, to me it seems we want to add a check for nulls that as far as possible can be picked up at compile time. I don't think we should change the nature of existing code nor introduce what seems to me to be areas of uncertainty. But like I said I honestly don't get it, I just keep seeing a cat in a box. If something at runtime has no value, what is it? Is it nothing, or is it something? This quantum stuff always does my head in ;)

@jdmichel
Copy link
Author

jdmichel commented Jan 9, 2019

My point about pattern matching being confusing is that VB already has several different concepts that could be called pattern matching.

  1. If x Like y Then
  2. Select Case
    Neither of these is suitable for the thing that I called TryCast Into, and I didn't understand why that concept would be known as pattern matching.

I googled and found Features of a pattern matching syntax, so I now understand how it relates.
What do you think of:

Select Case obj
Case TryCast Into f as Foo
    f.DoFoo()
Case TryCast Into b as Bar
    b.DoBar()
End Select

So within a Select Case the variable being tested is implied, but outside of a Select you would specify it.

If TryCast obj Into f As Foo Then
    f.DoFoo()
End If
Dim isMatch = TryCast obj Into f As Foo
Dim isNotNull = TryCast maybeNull Into nn As Bar

@jdmichel
Copy link
Author

jdmichel commented Jan 9, 2019

I'm really confused by the whole "cat in a box" thing.

I think maybe I see what you mean if you're pointing out that it's not clear what assigning Nothing to a non-nullable reference type should do. I think many people are probably confused that Nothing does not mean Null, and instead means Default, because in practice with nullable references the default could be Null. But what is the default for a NonNullable reference?

  1. It could be Null still, which would just make any attempt to assign it an error. However, that seem non-symettrical, and feels weird to define something illegal as the default.
  2. It could be a default constructed object, but that won't work since it breaks the expectation that Nothing is a single default so you couldn't test it with If x Is Nothing
  3. We could introduce some way to provide a default for each type, but that seems complicated to understand too, and may have all sorts of follow-on consequences.

The first seems easiest to me.

@Bill-McC
Copy link

Bill-McC commented Jan 9, 2019

A reference type at one stage or another is always null; if not it would be a value type.
So if we say a reference type is not nullable, we aren't actually talking about the type, rather a state the type can be in. And as much as the compiler can try to enforce it, by the very nature of the type, at some point it will be null. And the only way we can be sure is to open the box, to test for null.

@pricerc
Copy link

pricerc commented Jan 9, 2019

@pricerc

Although the discussion is nominally about nullable reference types; Is that not really a misnomer, since reference types are by definition nullable?

Not quite, although it is a little confusing. The C# proposal actually consists of two basic things:

1. **Reinterpret every current usage of a reference type as excluding `Nothing`**, so warnings can be issued on potential misuse; IOW, _reference types_ -> _non-nullable reference types_.

2. A syntax variation is now needed to define reference types that can include `Nothing`; this is the syntax that is being **added to the language -- a syntax for _nullable reference types_.** Hence, the name.

"tomato, tomato".

I read the blog. And I call it as I see it. They can spin it however they like, they're still talking about introducing a new concept of non-nullable reference types.

That they're wanting to gently shove C# people in that direction is fine. If C# enthusiasts are still struggling with the concept of null, then they probably need help :D.

I would suggest that we first have to consider what the right design should have been. ...

That's fine. But doing what C# is doing, which is introducing (even if opt-in) a breaking change to 20 years of legacy code, is simply not acceptable in the VB world. Which is why I would rather 'add' the feature as something that VB developers can choose to use. Or not.

... bundling Nothing as a valid value for reference types is a sort of violation of the contract implied by the type's shape.

Which is why I suggested that Nullable(Of T) should be extended to allow T to be a reference type. Which I think would then make all types, value or reference, have a common way of dealing with nullability.

...then saddling every future usage of reference types with an explicit annotation to denote non-nullability seems inappropriate.

Agreed. And being in VB-land, we'll make it an Option, and so maintain backward compatibility and allow our developers to move over at their own pace, as they deem appropriate in their SDLC.

my perception is that the VB.NET community is smaller, and less users who would be willing to pay this price.

I suspect VB developers are a little bit like many non-mainstream-thinkers in the world today - scared to raise their heads for fear of being called out for innappropriate language use, or that they'll be DOX'ed for being VB fanciers.

@pricerc
Copy link

pricerc commented Jan 9, 2019

To TL;DR my earlier textbook (apologies for that):

  1. Add a new option. I think Option NullRef|NullableReference Explicit|Implicit describes what I'm thinking.

    • Implicit is what we have today - references are by default nullable.
    • Explicit means just that - references are 'not nullable' by default, and must be explicitly marked as nullable when desired.
  2. Add annotations ! and ? to reference types

    • ! forces 'Not-nullable', regardless of which option is in effect, although it would be redundant (being the default) in 'Explicit' mode.
    • ? forces 'Nullable', regardless of which option is in effect, although it would be redundant (being the default) in 'Implicit' mode.
    • in the absence of annotation, the behaviour is determined by the option in effect.
  3. When a 'Not-nullable' parameter is specified in a method (or property), then the compiler should add a null check and ArgumentNullException.

By having this combination, you can code with or without using the options, and whichever convention suits your purposes.

This doesn't cover everything the original concept is going for, but I think it covers most of it, relatively simply.

Notes:

  • There are no breaking changes, unless you turn explicit mode on. Which could be a bit like turning Strict or Infer on for the first time.
  • Compared with some of the whizzy things the compiler guys have done, it strikes me that the incremental effort for this would be relatively small.
  • If Nullable(Of T) could be expanded to include reference types, it would make all types have consistent nullable behaviour.
  • I don't usually use the Nullable(Of T) syntax, but a NotNullable(Of T) companion might make sense.
  • this might obviate a key reason for Throw Expression  #370 - if arguments can't be null, there'd be no need for an exception expression when assigning to a local variable.

@jdmichel
Copy link
Author

jdmichel commented Jan 9, 2019

Maybe this will help. Let's try to understand what the dotnet clr looks like behind the scenes (simplified).
If you run the following VB code:
Dim r = new MyObject()
Then we now have several different pieces:

  1. There exists a reference type named MyObject
  2. There exists an object of that type.
  3. There exists a reference (aka pointer) to Consistency updates between C# and VB repos. #2 named "r"

Currently lets pretend that #3 looks something like this psuedocode:

Public Class NullableReference(Of T)
    Private myT As T

    Public Sub New(val As T)
        myT = val
    End Sub

    Public Property Value As T
        Get
            Return myT
        End Get
        Set(value As T)
            myT = value
        End Set
    End Property
End Class

What I'm trying to suggest is that we change the compiler to instead implement #3 as:

Public Class ImmutableReference(Of T)
    Private ReadOnly myT As T

    Public Sub New(val As T)
        myT = val
    End Sub

    Public ReadOnly Property Value As T
        Get
            Return myT
        End Get
    End Property
End Class

And I guess now that I write it out, there is an additional missing concept we haven't discussed which is whether non-nullable references should also be immutable.
If I understand the metaphor correctly, ImmutableReference ==> BoxedCat.

@Bill-McC
Copy link

Bill-McC commented Jan 9, 2019

The opposite. A reference type is mutable. It always starts as null. We can write special cases where we make it mutable only at instantiation, but that is a special case not a Type in the broad sense (as in ValueType, ReferenceType, Nullable ValueType etc).
What I was trying to get at is, no matter what you call it, the type is still a reference type and it can still be nullable or not. As Mads said,

There is no guaranteed null safety, even if you react to and eliminate all the warnings. There are many holes in the analysis by necessity, and also some by choice.

So it remains a cat in the box. (dead or not dead ?)

What we are interested in is null checking and safe code flow. This is what we practice today. We move away from the unknown state to a known state: we open the box and examine the cat.

So rather than pretend it be something it isn't I think we should just focus on those aspects. You can never guarantee it cannot be null, hence there is no such thing as a non nullable reference, and all reference types are nullable .

C# seems to be taking a shorthand approach, and annotating the type declaration to indicate if the compiler will raise warnings or not. The code can be compiled, warnings ignored, and a string?, string!, or string are still System.String, still all capable of being null. It is a "leaky abstraction"

If VB wanted to improve code flow checking at compile time, I think using attributes that say "warn on null", or similar would be a better fit. And being a compiler directive attribute it could be applied at field, parameter, variable, method, class, code file, maybe even project.

@KathleenDollard
Copy link
Contributor

@Bill-McC Are you suggesting more of an Assert strategy? With nice gestures to make it easy?

@Bill-McC
Copy link

@KathleenDollard not sure. I think more thorough compiler warnings/assertions, something it is easy for existing code to opt into, and something that makes it easy to opt out of at a granular level.
I definitely do not like the idea of pretending a reference type can't be null.
So if I want to have a way if declaring null checks, say on parameters on a method call, I don't see much value in declaring in the signature because ultimately that can and will result in a runtime null error that will be no easier to trace. So moving that into code block

@Bill-McC
Copy link

Moving into code block like an AssertNotNull(param, param, Action) or similar might help speed writing code.

@zspitz
Copy link

zspitz commented Jan 28, 2019

@Bill-McC

A reference type is mutable. It always starts as null. We can write special cases where we make it mutable only at instantiation, but that is a special case not a Type in the broad sense (as in ValueType, ReferenceType, Nullable ValueType etc).
What I was trying to get at is, no matter what you call it, the type is still a reference type and it can still be nullable or not. As Mads said,

There is no guaranteed null safety, even if you react to and eliminate all the warnings. There are many holes in the analysis by necessity, and also some by choice.

So it remains a cat in the box. (dead or not dead ?)

This is all true if the type system in VB.NET must precisely reflect the CLR type system. At the level of the CLR, a reference type always starts as null, and as long as it it mutable, can be set to null.

But what if we define a virtual type layer that exists only at the language level? Those types could have additional rules enforced by the compiler. It could be argued that VB.NET already does this -- extension methods can be called as if they were instance methods, even though they don't actually exist on the extended type.

It's true that because the CLR rules would be more lax, there is a greater potential for leaks in this abstraction; but that goes back to the point Mads made in the original post -- this kind of abstraction will never be perfect.

The example of Typescript is instructive here. If you want to be technical, Javascript does have a very primitive type system -- there is a single type enforced by the compiler/interpreter, to which language-defined keywords and operators can be applied. Trying to use arbitrary keywords/operators not defined by the language will be rejected by the interpreter. All of Typescript's types are simply a virtual layer on top of the "real" type system, one based more on runtime behavior than compilation.

@Bill-McC
Copy link

@zspitz if only at language level it would be even more prone to failure as it would fall down at every framework call.
The example of extension types, putting aside they fact they can only access non private members, they also have to deal with the instance they are referring to as being null. And this makes sense as often in LINQ the result of a query can be null. And so we have operators such as ?. simply because we can never do away with null, there is no such thing as a default customer.
The goal is not to do away with null checking, rather it is to help track down ppssible null exception sources at compile time. Ironically if you were to decorate a parameter as not allowing nulls, you are actually saying that code is not doing any null handling rather it may throw null exceptions.

@KathleenDollard
Copy link
Contributor

We won't be changing the type system for null-reference types (which, yes, is a bit of a misnomer).

It's hard to imagine that with C# picking the flow analysis route, and being quite happy with what they've achieved with it, that VB would take a route of a virtual type system.

@pricerc
Copy link

pricerc commented Jan 29, 2019

@ Bill-McC
if

Sub SomeMethod(value as MyClass!)
    ' do something
End Sub

is translated by the compiler into

Sub SomeMethod(value as MyClass)
    If value Is Nothing Then Throw New ArgumentNullException(NameOf(value))
End Sub

Then would the runtime error not then be in the 'right place' - in the method that doesn't want nulls?

@Bill-McC
Copy link

Bill-McC commented Jan 30, 2019

@pricerc yes it would be in the right place as a null check.

@zspitz
Copy link

zspitz commented Feb 17, 2020

@paul1956

The Type Name is "String Or Nothing" normally would be an error.

It isn't (tested on .NET Core 3.1, Option Strict On), because it's interpreted as:

If (TypeOf o Is String) Or Nothing Then
End If

and I'm guessing there's some sort of implicit conversion from the Nothing literal to Boolean, because the following also compiles:

If Nothing Then
End If

This is further supported by wrapping the expression in an Expression:

Dim expr As Expression(Of Func(Of Boolean)) = Function() TypeOf o Is String Or Nothing

which generates the following VB.NET string representation:

Function() TypeOf o Is String Or False

or in the DebugView of the expression:

.Lambda #Lambda1<System.Func`1[System.Boolean]>() {
    .Constant<_testOrType.Program+_Closure$__0-0>(_testOrType.Program+_Closure$__0-0).$VB$Local_o .Is System.String | False
}

@zspitz
Copy link

zspitz commented Feb 17, 2020

@paul1956 I don't follow how this:

One issue with Nullable String in VB is the If below returns True in VB but I believe False in C#.

Dim X as String = Nothing
If X = "" then
  Return True
Else
  Return False
End if

is relevant. If the variable X is identified as not allowing Nothing (either because of an Option or because it has some explicit annotation such as ! or NotNullable(Of String)), the compiler would warn on the first line attempting to assign Nothing.

If X is identified as allowing Nothing (again, either because of an Option, or because of some explicit annotation), the compiler wouldn't warn.

What does this have to do with VB.NET's evaluation of Nothing = "" as True?

Unless you're saying that If X = "" Then is used as a check for Nothing in VB.NET? But flow analysis would catch the Nothing on the previous line, as I noted.

@pricerc
Copy link

pricerc commented Feb 17, 2020

@zspitz

Let me rephrase, Mads Torgerson notes in the blog post introducing nullable reference types:

your code would look like it’s swearing at you, with punctuation? on! every? declaration!

Even more than C#, this applies to VB.NET with it's preference for an English-like syntax.

Some would argue that C# looks like it's swearing at you anyway, but I digress...

I wasn't arguing the merits (or lack thereof) of anything, just observing that "?" and "!" are existing programming shortcuts in VB, so their re-use in slightly different context would be just as 'VB-like' as their existing use.

Whether they should be overloaded for re-use in a different context is its own (related) discussion.

@pricerc
Copy link

pricerc commented Feb 17, 2020

@zspitz

Can you please expand on what you mean by "Forced Dereferencing"? I fear we may be talking at cross-purposes.

While there may be some esoteric academic discussion to be had, my interest in this conversation is the idea of having some declarative syntactic sugar to simplify existing code patterns; a compiler hint, if you will, that a variable (or parameter) intended to hold a reference type should not be null, and that I'd like the compiler to warn me and/or generate appropriate guard code as appropriate.

I can't help the feeling that you're talking about something slightly different.

Then, on "!": what you describe as a 'dictionary lookup', I think of as a 'dereference', because my only use of ! for this purpose has been limited to VBA in the context of MS Access, where I'm "dereferencing" the fields in an ADODB.RecordSet to get their values as if they were properties of the recordset. I don't believe I've ever used it in VB.Net.

I do think there should be a punctuation shortcut for explicitly declaring non-nullable reference types, but I don't have a strong opinion on whether "!" is a good one. Although, since I don't use it for anything at the moment, it would work well for me. As long as the intent is unambiguous from the context, I don't think it matters greatly.

But way more important than any discussion around punctuation shortcuts, is actually getting traction on the idea that differentiating between nullable and non-nullable references has a place in VB.

@AdamSpeight2008
Copy link
Contributor

Just to point out it isn't the type that is nullable but the identifier used to object of that type.
Dim example As SomeReferenceType , it is the example that is expressing the nullability.
In which it rules out ! as an indication of non-nullability as it currently as strict meaning as the type character for the value type Single. Dim value! = 0.12

I am in favor of using an attribute on the point where the variable is declared or introduced into scope. eg
Dim <NonNull>example As SomeReferenceType and
Public SomeMethod( <NonNull> example As SomeReferenceType )

@pricerc
Copy link

pricerc commented Feb 17, 2020

Just to point out it isn't the type that is nullable but the identifier used to object of that type.

I think (hope) we all know that.

I am in favor of using an attribute on the point where the variable is declared or introduced into scope.

two thoughts:

  1. Given that Nullable(Of T) is an inline declaration, I think it would be inconsistent for NotNull(Of T) to be an attribute instead. Nullable and NotNullable are (IMO) two different ways of treating the same concept, they have similar (if opposite) semantics, so should have a similar syntax.

  2. Using attributes for reference types when we use keywords for value types would introduce a difference in syntax between value and reference types. In an ideal world, the difference between value and reference types should be (almost) irrelevant to a VB programmer, the syntax and semantics for the two should be as similar as we can make it. One of the things I appreciated in the switch from VB6 to VB.Net was losing the 'Set' statement - this was a good thing. Introducing syntax which is different for value and reference types would be a step back.

@paul1956
Copy link

Given that 16.5.0 Preview 2 has a CodeFix to put all the guards in explicitly I am not sure this is that big an issue. Having the guards there explicitly make it clear what is happening. VB also already has the attribute which if it worked correctly would tell the compiler that an argument probably will be Null and it is definitely the methods responsibility to test before use (which could be hidden by compiler) and to set it. I think there would be a lot of internal changes to have a Type be 3 tokens an Identifier followed by 2 Keyword.

@pricerc
Copy link

pricerc commented Feb 17, 2020

Ok, playing devil's advocate again: If you're going to put the guards in, then what's the benefit to me as a programmer of a NotNull construct, whatever form it takes? I already use refactoring tools that will put the guard clauses in for me; I'm looking to de-clutter my code, not more-clutter it.

Borrowing from a comment I made in January last year:

' I'd like to see (roughly)
Sub SomeMethod(value as NotNullable(Of MyClass))
   ' do something
End Sub

' translated by the compiler into
Sub SomeMethod(value as MyClass)
   If value Is Nothing Then Throw New ArgumentNullException(NameOf(value))
  ' do something
End Sub

When coding, and I write a call to SomeMethod, the compiler and/or intellisense will let me know that value must not be null.

I think that would cover at least 90% of my potential uses for non-null references.

And the presence of NotNullable, or its punctuation-based shortcut, will tell me that there's a guard clause, it won't be any clearer to me if a code refactoring tool adds all the code in.

If you've got a method with several 'not null' parameters, but has only one line of actual code, being able to declare that a parameter needs guarding will make the method much easier to follow.

I get that attributes will also work, but as I mentioned, I think that would be inconsistent with how we handle Nullable(Of T).

Of course my actual preference would be my other suggestion, which is an option which reverses the 'default nullability' of reference types, making it more like C#'s implementation, so that by default reference variables are 'not null' and have to be declared Nullable in exactly the same way as value types need to be.

adapted from an earlier comment (leaving out most of it):

Option ReferenceNullable Explicit

Class Wine
	Public Property Name As String
	Public Property Vintage As Integer
End Class

' 1) value is 'NotNullable' because of option. 
Public Pub GetCaskImplicitNotNull(value as Wine) 
	' 1a) Compiler inserts null reference check here,
	'		throws ArgumentNullException(NameOf(value)) if value is Nothing
	Console.WriteLine($"{value.Name} {value.Vintage}")
End Sub

' 2) value is Nullable, semantics the same as today, and code analysis suggests guard clause.
Public Sub GetCaskExplicitNull(value as Nullable(Of Wine))                                           
	' 2a) RunTime NullRefException if value is null:
	Console.WriteLine($"{value.Name} {value.Vintage}")                        
End Sub
> 
Public Sub CallGetCasks
	Dim newWine As Wine = Nothing

	' Compiler warning, RunTime ArgumentNullException from compiler-added guard code
	GetCaskImplicitNotNull(newWine) 

	' No compiler warning, RunTime NullRefException (from called method)
	GetCaskExplicitNull(newWine) 

End Sub

@paul1956
Copy link

@pricerc Would I like the code cleaner YES and maybe the answer is to update the CodeFix to produce

    If AnyNull((value, NameOf(Value)), (value2, NameOf(Value2))(value3, NameOf(Value3))) Then
         ' Auto throw message below
         '    throws ArgumentNullException(NameOf(value)) if value is Nothing
    End If

Or with builti support for AnyNull

    If AnyNull(value, Value2, value 3) Then ' Auto throw message below
         '    throws ArgumentNullException(NameOf(valueX)) if value is Nothing
    End If

And the compiler produces something compatible with the 2 functions below

    Overridable Function AnyNull(ParamArray values() As (Object, String)) As Boolean
        Dim Msg As New System.Text.StringBuilder
        For Each value As (Object, String) In values
            If value.Item1 Is Nothing Then
                Throw New ArgumentException(value.Item2)
            End If
        Next
        Return False
    End Function

    Overridable Function AnyNull(Msg As StringBuilder, ParamArray values() As (Object, String)) As Boolean
        For Each value As (Object, String) In values
            If value.Item1 Is Nothing Then
                ' Build message
            End If
        Next
        If Msg.Length = 0 Then
            Return False
        End If
        Return True
    End Function
``

@paul1956
Copy link

The If is optional and it only needed if you pass a StringBuilder

@pricerc
Copy link

pricerc commented Feb 17, 2020

@paul1956

I appreciate the code fixes and where you're going, but at the same time, that's not introducing an enhancement to the VB language, it's an enhancement to 'standard' refactoring tools.

In fact, the same is true of suggestions involving the use of attributes instead of keywords - they're enhancing tooling, but not the language itself. Ironically, C# has stolen some of VB's thunder in that regard, e.g. with extension methods where they don't need to decorate them with attributes, and now possibly with (not-)nullable references as well.

@paul1956
Copy link

paul1956 commented Feb 18, 2020

You need an enhancement to the language to make it simple

AnyNull(value, Value2, value3)

vs below which can be done today

AnyNull((Value, NameOf(Value1), (Value, NameOf(Value2), (Value, NameOf(Value3))

AnyNull is a placeholder, a better name might be ThrowIfAnyNull and probably passing in Caller and line so the message point to the real issue vs the function doing the checking.

It seems C# makes (Public?) functions with the first parameter "this" extension methods. I am not sure that is as flexible as VB that can extend any Class from a module (static non-inheritable) , and C# might be able to do that as well. VB could do this same thing without breaking anything but I would not spend any time doing it.

@tverweij
Copy link

I did not see any indication in this thread of what a nullable reference means.
In principle, all references are nullable at this moment, causing a NullReferenceException when a reference is used while it is null.
As far as I am concerned, that last point is the problem.

So we are talking about nullable references, where we should talk about non-nullable references.
Because that would be new.

And what is a non-nullable reference?
In C# they chose for an implementation where the compiler is looking at chances that a reference could become null, but does not guarantee that a reference can not be null at any time.
It just issues warning about possible null references, but in the compiled version there is no difference between nullable and non-nullable references.

Besides the detection of possible null references, we could move the NullReferenceException to the assignment of a null value to a non-nullable reference; if you try to assign Null to such a reference, it means that the program does something illegal. This would move the exception from the first invocation of a method, property or field on a null reference to the point where the null reference is assigned.
With this behaviour, the chance that we will find the bug during testing grows exponentially. And when it occurs during release, we know one thing for sure; the problem lays in an assignment - making the effort needed to find the bug exponentially smaller.

@zspitz
Copy link

zspitz commented Feb 19, 2020

So we are talking about nullable references, where we should talk about non-nullable references.
Because that would be new.

Except that the C# design consists of two parts:

  1. Until now, reference types meant nullable reference types. Now, reference types are redefined to mean non-nullable reference types. This makes possible the flow analysis which warns about null assignments to places where the program expects it not to be null; but note that it doesn't require any new syntax.
  2. However, there certainly are places in our programs when we want to allow null to be assigned, and we want the rest of the program to be able to work with these variables/elements. This requires new syntax -- ? appended to the type name -- to be added to the language.

AFAICT that's why the name of this feature is nullable reference types -- because that is the only syntax being added to the language.

@tverweij
Copy link

Point is that we are talking VB here, not C#.
As stated a few times by someone from Microsoft: That something is available for another language is no reason to ask it for VB.

So I make it a VB issue, where the implementation can differ from C# as they are different languages.

@Happypig375
Copy link
Member

Happypig375 commented Feb 19, 2020

I'd say that VB should have a similar implementation as C#. VB is simple and beginner-friendly, but having to require ! or ? at every type declaration does not comply with simplicity. Moreover, requiring ! for reference types differ from the lack of ! on value types. Although these arguments are the same as C#, these are equally valid when applied on VB.

@tverweij
Copy link

@Happypig375
I agree.
But I also saw multiple times in this repository that breaking changes are by definition not granted.
So do as C# did might not be an option. We need a non breaking change.

@zspitz
Copy link

zspitz commented Feb 19, 2020

As stated a few times by someone from Microsoft: That something is available for another language is no reason to ask it for VB.

So I make it a VB issue, where the implementation can differ from C# as they are different languages.

Which isn't to say that a successful C# implementation could not be used in VB.NET, if nothing more than a starting point for further discussion,

And IMO the logical steps for the design decisions are the same for both languages:

  1. If VB.NET was being designed today, the ideal would be for elements typed as Random to not include null values -- first, because Random has a very different shape than Nothing, and virtually all of what you want to do with a Random cannot be done with a Nothing; and second, because otherwise there's a dichotomy between the type of an instance (which is never Nothing) and the type of a language element / expression (which currently could be Nothing).
  2. All else being equal, reinterpreting Random as a non-nullable reference type is therefore highly preferable to an explicit annotation.
  3. While this reinterpertation will raise warnings on existing code, this is mitigated by a) the feature is only opt-in for old code; and b) flow analysis can greatly reduce the number of warnings; leaving the vast majority of warnings as places where appropriate null guards should have been placed and are missing -- in other words, a high signal-to-noise ratio.
  4. There are certainly places where the warnings will be issued incorrectly; it's important to have some way of telling the compiler, "I know better, this is never going to be Nothing".

I think any proposals to diverge from the C# design needs to provide strong justification why VB should be different from C#.


Having said that, I could make a case that the syntax should not be the same:

  • Nullable annotation syntax in C# (Random?) aligns nicely with the syntax for nullable value types (Integer?). However, there is a difference -- nullable value types are actual types, down to the level of the CLR; while nullable annotations aren't reflected in the IL. Nullable Random won't have members such as Value and HasValue, and calling GetType() on a nullable Random instance will return the same as for a non-nullable Random instance.
    We could use attributes for this, but my personal preference is to define the element as Dim rnd As Random Or Nothing
  • I think that while a dedicated syntax in C# for forced dereferencing (Random!) is good, the number of places where it is necessary to do this is relatively small (after flow analysis). For VB.NET, which already has a tendency to use words instead of symbols, I think an attribute would be sufficient for these cases.

Some other random thoughts:

  • Block-level granularity? -- Why should this be different than any other Option which can be defined on the file or on the project, but not for an individual block or procedure? Why within the same file should I have Random in one place mean Random Or Nothing and in another place mean Random, not Nothing?
  • Breaking old code -- If, as in C#, this is implemented as a warning, code should compile just fine even after turning the option on. An alternative migration strategy would be to toggle it on the file level incrementally until all the warnings have been resolved across the project.
  • Resolving warnings -- It should be noted that resolving the warnings doesn't require all that much effort: a few places to indicate nullability, and a few places to indicate that I know better than the compiler and that something is not null, and flow analysis can do the rest.

@pricerc
Copy link

pricerc commented Feb 20, 2020

So we are talking about nullable references, where we should talk about non-nullable references.
Because that would be new.

Yes. I was sure this had been pointed out early in the thread, but I couldn't find it when I just looked.

So I make it a VB issue, where the implementation can differ from C# as they are different languages.

Absolutely. We should be making changes for the benefit of VB developers. By all means, see how other languages do things, but VB has a 'style' which should be maintained, so features being borrowed from other languages need to be adapted so that they make sense in VB, and look like VB.

Except that the C# design consists of two parts:

  1. Until now, reference types meant nullable reference types. Now, reference types are redefined to mean non-nullable reference types
    ...

and

But I also saw multiple times in this repository that breaking changes are by definition not granted.
So do as C# did might not be an option. We need a non breaking change.

in VB, we have the luxury of 'Option' statements, so we could introduce an option to turn this on or off, with the 'legacy' option being the default.

nullable value types are actual types, down to the level of the CLR; while nullable annotations aren't reflected in the IL. Nullable Random won't have members such as Value and HasValue, and calling GetType() on a nullable Random instance will return the same as for a non-nullable Random instance

Why shouldn't Nullable Random have members such as Value and HasValue?

If we're talking about a VB-specific feature, I have seen no compelling reason why VB couldn't make Nullable(Of ReferenceType) behave the same way as Nullable(Of ValueType) - and provide HasValue and Value properties (note, I'm not making a judgement on the complexity of doing so). I've often wondered why reference types don't have an implicit implementation exactly like that, so that instead of Is rnd Nothing we could use If rnd.HasValue.

Besides the detection of possible null references, we could move the NullReferenceException to the assignment of a null value to a non-nullable reference;

I think this is a major benefit, along with removing the need for manually coded guard clauses in methods. I.e. if the 'non-nullable references' option is turned on, then:

Public Sub Routine(rnd As Random)
	'do something
End Sub

could be compiled as if you'd coded:

Public Sub Routine(rnd As Random)
    If rnd Is Nothing Then Throw New ArgumentNullException(NameOf(rnd)) ' or NullReferenceException
	'do something
End Sub

Additionally, if the method is private, an optimizer could move the guard clause to the calling method.

@tverweij
Copy link

in VB, we have the luxury of 'Option' statements, so we could introduce an option to turn this on or off, with the 'legacy' option being the default.

That is absolutely a good idea.

So we get (sorry @zspitz I really don't like the Or Nothing syntax; a type is one word in VB):

#Option ReferenceNotNullable On
Dim NonNullableObject As Object = MyDefault 'Has to be initialized as it can not be null
Dim NullableObject As Object?

or

#Option ReferenceNotNullable Off 'This is the default to not break any code
Dim NonNullableObject As Object! = MyDefault 'Has to be initialized as it can not be null
Dim NullableObject As Object

If you assign a nullable object to a non nullable object:

NonNullableObject = NullableObject

It is compiled as:

If NullableObject is Nothing Then Throw new ArgumentNullException(NameOf(NullableObject ))
NonNullableObject  = NullableObject

When a function result is assigned to a nullable object, it needs an extra variable in the resulting code:

Dim _tmp = CallToNullableFunction()
If _tmpis Nothing Then Throw new ArgumentNullException(NameOf(CallToNullableFunction))
NonNullableObject  = _tmp```

But to be able to write rubust code, we should add an overload to TryCast:

Function TryCast(TypeToCast As Object, TypeToCastTo as Type, FailureValue as Object!)

This call differs from the existing TryCast that it won't return nothing, but a defined non nullable FailureValue instead.

So we can assign like this (with Option ReferenceNotNullable Off):

NonNullableObject = TryCast(NullableObject, Object!, MyDefaultValue)

This will make sure that the assignment will always succeed; when NullableObject is Nothing, the nonnullable reference MyDefaultValue will be assigned.

@tverweij
Copy link

Why shouldn't Nullable Random have members such as Value and HasValue?
If we're talking about a VB-specific feature, I have seen no compelling reason why VB couldn't make Nullable(Of ReferenceType) behave the same way as Nullable(Of ValueType) - and provide HasValue and Value properties (note, I'm not making a judgement on the complexity of doing so).

That one is simple, just add the following extension methods:

Module Module1

    <Runtime.CompilerServices.Extension()>
    Function HasValue(anObject As Object) As Boolean
        Return anObject IsNot Nothing
    End Function

    <Runtime.CompilerServices.Extension()>
    Function Value(anObject As Object) As Object
        Return anObject
    End Function

End Module

@zspitz
Copy link

zspitz commented Feb 20, 2020

@tverweij

So we get (sorry @zspitz I really don't like the Or Nothing syntax; a type is one word in VB)

But that's precisely the point of the syntax. The C# implementation does not create different types for nullable and non-nullable references; it's simply an annotation, and could easily be replaced with an attribute. Having the same syntax for "two things which are used the same way, and have more or less the same behavior" might be OK for C# (which uses the : for both inheriting classes and implementing interfaces). But if there are significant differences in behavior between the two (e.g. Value and HasValue properties, different results from GetType() calls, variations in generics usage) then I think it important to emphasize that this is not part of the type.

@pricerc
Copy link

pricerc commented Feb 20, 2020

Having the same syntax for "two things which are used the same way, and have more or less the same behavior" might be OK for C#

I would argue that it should be OK for VB too.

When the move from VB6 to VB.Net happened, we got rid of Let and Set, because they were two different ways of doing the same thing: assigning a value to a variable. The idea of extending Nullable(Of T) to reference types is just an extension of this existing idea.

Your example (: for inherits/implements) is a quite different to the discussion on reference vs value types, and fails to offer a compelling reason why VB couldn't or shouldn't make Nullable(Of ReferenceType) behave the same way as Nullable(Of ValueType).

it's simply an annotation, and could easily be replaced with an attribute

Attributes are 'compiler hints', not language enhancements. Given the maturity of VB, many of the new concepts that get proposed for it could probably be implemented as attributes, but that wouldn't make them enhancements to the language. And surely the point of this vblang forum is to discuss potential enhancements to the VB language? I don't think that the fact that you can do something with an attribute is a good reason on its own to write off making an enhancement to the language itself.

Not that I like talking about C# on the VB forum, but since you raised it, and extension methods in C# have already come up in this discussion - IIRC, when extension methods first happened, C# used attributes to enable them, just as VB still does. Except they later figured out a revised language syntax to handle this fairly popular construct, and dispensed with the clutter of the attribute.

Taking a step back, and looking at design choices you'd make if you were starting from scratch with a 'new' VB. Would you choose to have two different types of structured type (classes and structures), or would you choose just one at let the compiler and run-time choose whether to put things on the heap or on the stack? If it were me, I'd choose the latter; it would simplify the language, since in most cases, on modern computers, with a modern O/S, it probably doesn't matter which you choose. Many languages don't offer the choice, for the good reason that it shouldn't matter to your common-or-garden variety software developer. Perhaps an expert doing fine-grained tuning of a problematic piece of code would have reason to specifically choose heap vs stack, but then perhaps that where attributes would work well, as compiler hints, for experts.

@tverweij
Copy link

I just did a test with C# (non) nullable types.
In the compiled binary, there is NO difference between a nullable reference and a non nullable reference. The generated IL is 100% identical.
And because the runtime is build in C#, I have to conclude that VB can also no do anything to make them different because that would make VB incompatible with the runtime.

So - whatever we do - it will be about syntaxial sugar and / or checking code generation.

@zspitz
Copy link

zspitz commented Feb 21, 2020

@pricerc

I would argue that it should be OK for VB too.

AFAICT you are proposing two separate things:

  1. allow System.Nullable(Of T) to take a reference type as a generic parameter (currently it only allows value types).
  2. use the T? syntax as a shorthand for Nullable(Of T); since per 1 Nullable(Of T) could use both reference- and value-types, the shorthand would work for both.

But in C#, nullable references are not a new type; they are nothing more than a compiler annotation. In that case, is the T? shorthand still appropriate, considering that <reference-type>? is at the CLR level exactly the same as <reference-type>, and very different from <value-type>? or Nullable(Of ValueType)?

Attributes are 'compiler hints', not language enhancements.

Absolutely. But an attribute could certainly be used to power language enhancements; for example, because a method has the <Extension> attribute applied, it can now be called as if it was an instance methods, which is a language enhancement. Having an attribute to describe nullability enables flow analysis to determine if subsequent assignment or other operations are appropriate.

NB. There's also a balance in creating a new syntax. Part of the difficulty of implementing extension everything in C# is because the syntax used for methods can't be easily applied to other members.

fails to offer a compelling reason why VB couldn't or shouldn't make Nullable(Of ReferenceType) behave the same way as Nullable(Of ValueType).

I'm still thinking about this, and haven't come to a final conclusion.

@tverweij

And because the runtime is build in C#, I have to conclude that VB can also no do anything to make them different because that would make VB incompatible with the runtime.

Not really. As long as the VB compiler could reduce it to a CTS type, the CLR should handle it just fine.

@zspitz
Copy link

zspitz commented Feb 21, 2020

@pricerc

Would you choose to have two different types of structured type (classes and structures), or would you choose just one at let the compiler and run-time choose whether to put things on the heap or on the stack?

I don't think that's really practical. The primary distinction between classes and structures is whether instances of the type must be transferred by reference (classes) or can the value be transferred (structures). If I've written my program in such a way that it depends on one or the other, and then I modify my type causing the compiler to switch it from a structure to a class or vice versa, my program's behavior will change without any other indication.

Thus, there is an explicit syntax difference between the definition of a class and a structure, because it affects the behavior of -- and my ability to reason about -- code that uses the type.

@McZosch
Copy link

McZosch commented Jun 5, 2021

But in C#, nullable references are not a new type; they are nothing more than a compiler annotation. In that case, is the T? shorthand still appropriate, considering that <reference-type>? is at the CLR level exactly the same as <reference-type>, and very different from <value-type>? or Nullable(Of ValueType)?

And this is indeed the "wilderness" in the C# approach. And it is not a really good one.

Rust handles null exactly like Nullable<T>. It's a bit-flag on a value type. So, opening Nullable<T> for ref types plus properly changing type definitions in the BCL is exactly what would be required. It would ask for no arkward compiler magic. It would support any language. It would instantly be supported by reflection, and we wouldn't need helper extensions being developed in two signatures.

This would indeed be introducing breaking changes in the BCL. I tend to think, VB would handle this quite well, due to Nothing having the meaning of default. Since C# has itself gotten a default keyword, the upgrade from assiging default instead of null should be quite as easy. This could also be done implicitly by a compiler switch, and the code still compiles.

And yes, default of String should return String.Empty. In fact, default should be an operator.

I get the feeling, that this C#-driven development of the BCL is doing more harm than it adds valuable features. This is indeed a threath to the whole ecosystem. More and more, I understand Kathleens conservativeness.

That said, I would still want diamond lambdas in VB.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests