Skip to content

Commit

Permalink
Rename Pricing to Price in Coster
Browse files Browse the repository at this point in the history
  • Loading branch information
gabrielweyer committed Oct 26, 2024
1 parent 6e3ef17 commit 3194b48
Show file tree
Hide file tree
Showing 21 changed files with 144 additions and 145 deletions.
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,22 +6,22 @@ Mass-pricing of VMs on Azure based on CPU cores count and memory. This is useful

This tool is composed of two components:

1. A [Parser](#parser) retrieving the pricing from [Virtual Machines Pricing][virtual-machines-pricing]
1. A [Parser](#parser) retrieving the prices from [Virtual Machines Pricing][virtual-machines-pricing]
2. A [Coster](#coster) using the output from the `Parser` and a list of VM specifications to determine their price

This approach allows to decouple pricing acquisition from its usage and open the door to automation. The `Parser` can be scheduled to retrieve the pricing at regular interval and the `Coster` can then use an always up-to-date pricing.
This approach allows to decouple price acquisition from its usage and open the door to automation. The `Parser` can be scheduled to retrieve the prices at regular interval and the `Coster` can then use always up-to-date prices.

[![Build Status][github-actions-parser-shield]][github-actions-parser] ([tests documentation](./docs/parser-tests.md))

[![Build Status][github-actions-coster-shield]][github-actions-coster]

## Parser

Retrieve VMs **hourly pricing** for a specific combination of **culture**, **currency**, **operating system** and **region**.
Retrieve VMs **hourly prices** for a specific combination of **culture**, **currency**, **operating system** and **region**.

:rotating_light: the parser is not - yet - able to retrieve pricing for the regions `east-china2`, `north-china2`, `east-china` and `north-china` as it is available on a [different website][azure-china].
:rotating_light: the parser is not - yet - able to retrieve prices for the regions `east-china2`, `north-china2`, `east-china` and `north-china` as it is available on a [different website][azure-china].

:rotating_light: the parser is not able to retrieve pricing for the regions `us-dod-central` and `us-dod-east` as no virtual machines are listed as publicly available.
:rotating_light: the parser is not able to retrieve prices for the regions `us-dod-central` and `us-dod-east` as no virtual machines are listed as publicly available.

### Parser pre-requisites

Expand Down Expand Up @@ -106,15 +106,15 @@ docker run --rm -it -v ./data:/data/ azure-vm-pricing:latest bash -c "yarn crawl

## Coster

Price VMs using the `JSON` pricing files generated by the `Parser`. The `Coster` will select the cheapest VM that has enough CPU cores and RAM.
Price VMs using the `JSON` prices files generated by the `Parser`. The `Coster` will select the cheapest VM that has enough CPU cores and RAM.

### Coster pre-requisites

- [.NET SDK 8.x][dotnet-sdk]

### Coster usage

You should paste the `JSON` pricing files generated by the `Parser` in the `coster\src\AzureVmCoster\Pricing\` folder. Setting the `culture` is only relevant when dealing with pricing and input files that were written using another culture with a different decimal point (e.g. comma vs period).
You should paste the `JSON` prices files generated by the `Parser` in the `coster\src\AzureVmCoster\Prices\` folder. Setting the `culture` is only relevant when dealing with prices and input files that were written using another culture with a different decimal point (e.g. comma vs period).

In `Release` mode:

Expand Down
6 changes: 3 additions & 3 deletions coster/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
bin/
obj/

# Pricing
src/AzureVmCoster/Pricing/*.json
src/AzureVmCoster/Pricing/*.csv
# Prices
src/AzureVmCoster/Prices/*.json
src/AzureVmCoster/Prices/*.csv

# Build
artifacts/
Expand Down
2 changes: 1 addition & 1 deletion coster/src/AzureVmCoster/AzureVmCoster.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
</ItemGroup>

<ItemGroup>
<None Update="Pricing/**">
<None Update="Prices/**">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
Expand Down
4 changes: 2 additions & 2 deletions coster/src/AzureVmCoster/Models/FileIdentifier.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ internal class FileIdentifier
{
public string Region { get; }
public string OperatingSystem { get; }
public string PricingFilename => $"vm-pricing_{Region}_{OperatingSystem}.json";
public string PriceFilename => $"vm-pricing_{Region}_{OperatingSystem}.json";

public FileIdentifier(string region, string operatingSystem)
{
Expand All @@ -23,7 +23,7 @@ public static FileIdentifier From(FileInfo fileInfo)
extensionIndex < underscoreLastIndex)
{
throw new ArgumentOutOfRangeException(nameof(fileInfo), fileInfo.Name,
"The pricing filename does not follow the pattern 'vm-pricing_<region>_<operating-system>.json'");
"The price filename does not follow the pattern 'vm-pricing_<region>_<operating-system>.json'");
}

var region = fileInfo.Name.Substring(underscoreFirstIndex + 1, underscoreLastIndex - underscoreFirstIndex - 1);
Expand Down
34 changes: 17 additions & 17 deletions coster/src/AzureVmCoster/Models/PricedVm.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,29 @@ namespace AzureVmCoster.Models;

internal class PricedVm
{
public PricedVm(InputVm inputVm, VmPricing? vmPricing)
public PricedVm(InputVm inputVm, VmPrice? vmPrice)
{
Name = inputVm.Name;
Region = inputVm.Region;
OperatingSystem = inputVm.OperatingSystem;

if (vmPricing != null)
if (vmPrice != null)
{
Instance = vmPricing.Instance;
VCpu = vmPricing.VCpu;
Ram = vmPricing.Ram;
PayAsYouGo = vmPricing.PayAsYouGo;
PayAsYouGoWithAzureHybridBenefit = vmPricing.PayAsYouGoWithAzureHybridBenefit;
OneYearReserved = vmPricing.OneYearReserved;
OneYearReservedWithAzureHybridBenefit = vmPricing.OneYearReservedWithAzureHybridBenefit;
ThreeYearReserved = vmPricing.ThreeYearReserved;
ThreeYearReservedWithAzureHybridBenefit = vmPricing.ThreeYearReservedWithAzureHybridBenefit;
Spot = vmPricing.Spot;
SpotWithAzureHybridBenefit = vmPricing.SpotWithAzureHybridBenefit;
OneYearSavingsPlan = vmPricing.OneYearSavingsPlan;
OneYearSavingsPlanWithAzureHybridBenefit = vmPricing.OneYearSavingsPlanWithAzureHybridBenefit;
ThreeYearSavingsPlan = vmPricing.ThreeYearSavingsPlan;
ThreeYearSavingsPlanWithAzureHybridBenefit = vmPricing.ThreeYearSavingsPlanWithAzureHybridBenefit;
Instance = vmPrice.Instance;
VCpu = vmPrice.VCpu;
Ram = vmPrice.Ram;
PayAsYouGo = vmPrice.PayAsYouGo;
PayAsYouGoWithAzureHybridBenefit = vmPrice.PayAsYouGoWithAzureHybridBenefit;
OneYearReserved = vmPrice.OneYearReserved;
OneYearReservedWithAzureHybridBenefit = vmPrice.OneYearReservedWithAzureHybridBenefit;
ThreeYearReserved = vmPrice.ThreeYearReserved;
ThreeYearReservedWithAzureHybridBenefit = vmPrice.ThreeYearReservedWithAzureHybridBenefit;
Spot = vmPrice.Spot;
SpotWithAzureHybridBenefit = vmPrice.SpotWithAzureHybridBenefit;
OneYearSavingsPlan = vmPrice.OneYearSavingsPlan;
OneYearSavingsPlanWithAzureHybridBenefit = vmPrice.OneYearSavingsPlanWithAzureHybridBenefit;
ThreeYearSavingsPlan = vmPrice.ThreeYearSavingsPlan;
ThreeYearSavingsPlanWithAzureHybridBenefit = vmPrice.ThreeYearSavingsPlanWithAzureHybridBenefit;
}
else
{
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
namespace AzureVmCoster.Models;

public class VmPricing
public class VmPrice
{
public string Region { get; set; } = default!;
public string OperatingSystem { get; set; } = default!;
Expand Down
File renamed without changes.
4 changes: 2 additions & 2 deletions coster/src/AzureVmCoster/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ public static async Task<int> Main(string[] args)
{
var builder = Host.CreateDefaultBuilder(args);
builder.ConfigureServices(s => s
.AddSingleton(new PriceDirectory(@"Pricing\"))
.AddSingleton(new PriceDirectory(@"Prices\"))
.AddSingleton<Pricer>()
.AddSingleton<ArgumentReader>()
.AddSingleton<VmPricingParser>()
.AddSingleton<VmPriceParser>()
.AddSingleton<PricedVmWriter>()
.AddSingleton<PriceService>());
var host = builder.Build();
Expand Down
8 changes: 4 additions & 4 deletions coster/src/AzureVmCoster/Services/PriceService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ namespace AzureVmCoster.Services;
internal class PriceService
{
private readonly Pricer _pricer;
private readonly VmPricingParser _vmPricingParser;
private readonly VmPriceParser _vmPriceParser;
private readonly PricedVmWriter _pricedVmWriter;

public PriceService(Pricer pricer, VmPricingParser vmPricingParser, PricedVmWriter pricedVmWriter)
public PriceService(Pricer pricer, VmPriceParser vmPriceParser, PricedVmWriter pricedVmWriter)
{
_pricer = pricer;
_vmPricingParser = vmPricingParser;
_vmPriceParser = vmPriceParser;
_pricedVmWriter = pricedVmWriter;
}

Expand All @@ -18,7 +18,7 @@ public async Task PriceAsync(string? inputFilePath, string? configurationFilePat
var inputFile = InputFileValidator.Validate(inputFilePath);
var inputVms = InputVmParser.Parse(inputFile, culture);

var vmPrices = await _vmPricingParser.ParseAsync();
var vmPrices = await _vmPriceParser.ParseAsync();

var configuration = await CosterConfiguration.FromPathAsync(configurationFilePath);

Expand Down
18 changes: 9 additions & 9 deletions coster/src/AzureVmCoster/Services/Pricer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ public Pricer(ILogger<Pricer> logger)
_logger = logger;
}

public List<PricedVm> Price(IList<InputVm> inputVms, IList<VmPricing> vmPrices, CosterConfiguration configuration)
public List<PricedVm> Price(IList<InputVm> inputVms, IList<VmPrice> vmPrices, CosterConfiguration configuration)
{
EnsurePricingExists(inputVms, vmPrices);
EnsurePriceExists(inputVms, vmPrices);

var filteredVmPrices = FilterPrices(vmPrices, configuration.ExcludedVms);

Expand All @@ -30,24 +30,24 @@ public List<PricedVm> Price(IList<InputVm> inputVms, IList<VmPricing> vmPrices,
var minCpu = vm.Cpu > 0 ? vm.Cpu : medianCpu;
var minRam = vm.Ram > 0 ? vm.Ram : medianRam;

var pricing = orderedVmPrices.FirstOrDefault(p =>
var price = orderedVmPrices.FirstOrDefault(p =>
p.Region.Equals(vm.Region, StringComparison.Ordinal) &&
p.OperatingSystem.Equals(vm.OperatingSystem, StringComparison.Ordinal) &&
p.Ram >= minRam &&
p.VCpu >= minCpu);

if (pricing == null)
if (price == null)
{
_logger.LogWarning("Could not find a matching pricing for VM '{VmName}' ({VmCpu} CPU cores and {VmRam} GB of RAM)", vm.Name, vm.Cpu, vm.Ram);
_logger.LogWarning("Could not find a matching price for VM '{VmName}' ({VmCpu} CPU cores and {VmRam} GB of RAM)", vm.Name, vm.Cpu, vm.Ram);
}

pricedVms.Add(new PricedVm(vm, pricing));
pricedVms.Add(new PricedVm(vm, price));
}

return pricedVms;
}

private static void EnsurePricingExists(IList<InputVm> vms, IList<VmPricing> vmPrices)
private static void EnsurePriceExists(IList<InputVm> vms, IList<VmPrice> vmPrices)
{
var missingFiles = vms
.Select(vm => new FileIdentifier(vm.Region, vm.OperatingSystem))
Expand All @@ -59,7 +59,7 @@ private static void EnsurePricingExists(IList<InputVm> vms, IList<VmPricing> vmP

if (missingFiles.Count > 0)
{
throw new InvalidOperationException($"Pricing files are missing for {JsonSerializer.Serialize(missingFiles)}");
throw new InvalidOperationException($"Price files are missing for {JsonSerializer.Serialize(missingFiles)}");
}
}

Expand All @@ -71,7 +71,7 @@ private static void EnsurePricingExists(IList<InputVm> vms, IList<VmPricing> vmP
/// <param name="vmPrices">The list of prices to filter</param>
/// <param name="excludedVms">The list of instances to remove</param>
/// <returns>The filtered prices</returns>
private static List<VmPricing> FilterPrices(IList<VmPricing> vmPrices, IList<string> excludedVms)
private static List<VmPrice> FilterPrices(IList<VmPrice> vmPrices, IList<string> excludedVms)
{
return vmPrices.Where(p => !excludedVms.Contains(p.Instance, StringComparer.OrdinalIgnoreCase)).ToList();
}
Expand Down
40 changes: 40 additions & 0 deletions coster/src/AzureVmCoster/Services/VmPriceParser.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
namespace AzureVmCoster.Services;

internal class VmPriceParser
{
private readonly string _priceDirectory;

public VmPriceParser(PriceDirectory priceDirectory)
{
_priceDirectory = priceDirectory.Directory;
}

public async Task<IList<VmPrice>> ParseAsync()
{
var priceFiles = Directory.GetFiles(_priceDirectory, "*.json");

var allVmPrices = new List<VmPrice>();

foreach (var priceFile in priceFiles)
{
var fileIdentifier = FileIdentifier.From(new FileInfo(priceFile));

var fileVmPrices = await JsonReader.DeserializeAsync<List<VmPrice>>(priceFile);

if (fileVmPrices == null || fileVmPrices.Count == 0)
{
continue;
}

fileVmPrices.ForEach(price =>
{
price.Region = fileIdentifier.Region;
price.OperatingSystem = fileIdentifier.OperatingSystem;
});

allVmPrices.AddRange(fileVmPrices);
}

return allVmPrices;
}
}
41 changes: 0 additions & 41 deletions coster/src/AzureVmCoster/Services/VmPricingParser.cs

This file was deleted.

4 changes: 2 additions & 2 deletions coster/tests/AzureVmCosterTests/Models/FileIdentifierTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ namespace AzureVmCosterTests.Models;
public static class FileIdentifierTests
{
[Fact]
public static void GivenInitialisedIdentifier_WhenGetPricingFilename_ThenExpected()
public static void GivenInitialisedIdentifier_WhenGetPriceFilename_ThenExpected()
{
// Arrange
var identifier = new FileIdentifier("some-region", "some-operating-system");

// Actual
var actual = identifier.PricingFilename;
var actual = identifier.PriceFilename;

// Assert
actual.Should().Be("vm-pricing_some-region_some-operating-system.json");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ public void GivenEmptyOrNullFilePath_ThenThrow(string? filePath)
public void GivenNonCsvExtension_ThenThrow()
{
// Arrange
const string filePath = "TestFiles/Pricing/vm-pricing_germany-west-central_windows.json";
const string filePath = "TestFiles/Price/vm-pricing_germany-west-central_windows.json";

// Act
Assert.Throws<ArgumentOutOfRangeException>(() => InputFileValidator.Validate(filePath));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ public void GivenPricedVm_WhenWrite_ThenPopulateAllColumns()
{
// Arrange
var inputVm = InputVmBuilder.AsUsWestWindowsD2V3Equivalent();
var vmPricing = VmPricingBuilder.AsUsWestWindowsD2V3();
var vm = PricedVmBuilder.From(inputVm, vmPricing);
var vmPrice = VmPriceBuilder.AsUsWestWindowsD2V3();
var vm = PricedVmBuilder.From(inputVm, vmPrice);
var fileName = $"{Guid.NewGuid():D}.csv";

// Act
Expand Down
Loading

0 comments on commit 3194b48

Please sign in to comment.