PC-1344: Update C# and .NET

[samyou-softwire]

Link to Jira ticket

Description

follow on from PC-1049 #349, this ticket updates C# and .NET versions for the app.

following our initial findings, this ticket also moves the eligible postcodes to a JSON file where they are loaded at runtime

Checklist

• I have made any necessary updates to the documentation
• I have checked there are no unnecessary IDE warnings in this PR
• I have checked there are no unintentional line ending changes
• I have added tests where applicable
• If I have made any changes to the code, I have used the IDE auto-formatter on it
• If I have made any changes to website flow, I have updated the Flow Miro Board
• If I have made any changes to website flow, I have checked forward and back behaviour is still consistent
• If I have made any changes to the Local Authority or Consortium data, I have made sure these changes have been reflected in the HUG2 Portal repository
• If I have made any changes to the ReferralRequest model, I have made sure the data in FakeReferralGenerator.cs is still accurate

Actions on Merge

• Check that the CLI still works (it was added after the initial ticket)

[samyou-softwire]

favour throwing an error in the case that this one doesn't exist

[Glenn-Clarke]

Do we want to still upgrade(?) these dependencies as part of this ticket? I think this might be a remnant of the original troubleshooting

[samyou-softwire]

have reverted; will see if pipeline still works fine

[Glenn-Clarke]

Here as well

[jdgage]

Do we need both of these? Have you tested without?

[samyou-softwire]

seems not

[jdgage]

Reminder to come back to this if we still need to later: it feels brittle to have the paths hard-coded here and I think we'd want a better way to tell if we're running the app locally than checking if the file exists at that path.

[jdgage]

I'm not sure about this approach. We've got

services.AddScoped<IEligiblePostcodeService, EligiblePostcodeService>();

in Startup.cs, which I believe means we'll get a fresh instance of this class per request. I'm concerned about the memory demands if we're reading the JSON and storing in memory for each instance. Previously this wouldn't have been an issue because the list was a static property, so we only ever had one copy in memory.

If we're keeping the postcodes in the codebase, we'd like to have a single copy of the data pulled into memory at build-time, or otherwise when first needed. But that's still potentially quite a load. Did you think about storing them in the database?

This is the sort of thing that it's probably worth getting wider opinions on around Softwire, for what it's worth. I'm happy to proof-read the question before you ask it if you like: I expect the way you ask it will have quite a big impact on how helpful the answers are.

[samyou-softwire]

for what it's worth the full length of the postcode JSON file is 5.5mb, so I wasn't super convinced this would be too taxing on application memory too much

agree the approach could be improved. I did a bit of research into including the data at build time from a JSON though couldn't find much online about it, other than including the file and loading it an runtime. looks like AddSingleton would make it so a single instance is created https://learn.microsoft.com/en-us/dotnet/api/Microsoft.Extensions.DependencyInjection.ServiceLifetime?view=net-8.0

could do a database col for this? though including the data in the database automatically sounds difficult, since we either need to load the json when running the migration (hard to test) or include it in the migration code (which could lead to the same issue as before)

can draft up a question if you'd like, just want to be clear on what our constrains and considerations are here

[jdgage]

So yes, we can have a singleton service. I think we'd want to extract just the postcodes into that service, rather than the whole thing.

Database table we'd set up via migration: I wouldn't expect it to cause the same issue, but I suppose it's possible we might have to be a bit creative with the migration - not a guarantee.

It's entirely possible the memory impact isn't really something to worry about - I don't have a great heuristic for this. That's one of the things you could ask about. Will want to think about scalability though + the difficulty is that we don't have a good way to test out if it would be an issue before releasing.

[samyou-softwire]

We have a part of our ASP.NET service that checks if a users' postcode is included in a fixed list of eligible postcodes. There are a large amount of postcodes to compare against (~400,000) and this check is done once per user.

Previously, the postcode list was included in a .cs file as part of the project, however we ran into issues where this file took a non-significant amount of time to compile, so we've had to rethink our approach.

The two main approaches we're considering are the following:

• Extracting the postcodes into a JSON file and loading this at runtime. We'd load the data into application memory once at startup (using a Singleton service) and check if a postcode is in this list
• Creating a new database table to hold our postcodes. This would be populated with the new data using a migration

Wonder what the thoughts are on the trade-offs between these two approaches? The database would certainly be more scalable though wonder for the amount of data being queried here whether it would justify the additional required setup?

[jdgage]

Suggested change

	eligiblePostcodes = JsonConvert.DeserializeObject<List<string>>(jsonContents);
	Previously, the postcode list was included in a .cs file as part of the project, however since upgrading to .NET 8 this makes the project take a significant amount of time (~50min) to compile, so we've had to rethink our approach.

Otherwise I might be tempted to not suggest approaches and instead just ask what people would suggest? Can come in later with what we thought of.