Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New storage proposal settings #656

Merged
merged 92 commits into from
Aug 29, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
e93e79e
Remove dead code
ancorgs May 29, 2023
bf8a1c8
WIP: very preliminary steps to ProposalSettings
ancorgs Jul 10, 2023
ed138ce
WIP: draft for VolumeOutline (instead of templates)
ancorgs Jul 17, 2023
93cd4b1
WIP: D-Bus conversions
joseivanlopez Jul 18, 2023
c3ac9e8
WIP: draft for getting the volume template for a path
ancorgs Jul 19, 2023
11baf61
WIP: D-Bus conversions
joseivanlopez Jul 19, 2023
69c3984
WIP: use VolumeTemplatesBuilder
joseivanlopez Jul 20, 2023
ff69860
WIP: adapt proposal backend
joseivanlopez Jul 20, 2023
aad83cb
WIP: add ProposalSettingsReader
joseivanlopez Jul 20, 2023
d015f34
WIP: Add system_vg_devices
joseivanlopez Jul 21, 2023
88c3309
WIP: to_y2storage conversion
joseivanlopez Jul 21, 2023
f2ad77b
WIP: add missing dbus data
joseivanlopez Jul 21, 2023
32c8090
WIP: remove format_options
joseivanlopez Jul 21, 2023
b233c4c
WIP: minor corrections and a temporary test
ancorgs Jul 21, 2023
88e986a
WIP: some improvements
joseivanlopez Jul 21, 2023
102ab2d
WIP: conversion from y2storage
joseivanlopez Jul 21, 2023
23bac3e
WIP: some fixes
joseivanlopez Jul 21, 2023
cf4e693
WIP: doc code and small fixes
joseivanlopez Jul 24, 2023
7dc224e
WIP: Rubocop
joseivanlopez Jul 24, 2023
9b3a7e0
WIP: Rubocop
joseivanlopez Jul 24, 2023
86b9ef1
Several small fixes
ancorgs Jul 25, 2023
6469521
WIP: reorganizing tests and adding temporary fixes
ancorgs Jul 26, 2023
2f41a60
WIP: more small adaptations
ancorgs Jul 26, 2023
035f4f0
WIP: some fixes
joseivanlopez Jul 26, 2023
bdd1c1a
Tests for conversions to D-Bus
joseivanlopez Jul 26, 2023
aca34ed
Test settings conversion from D-Bus
joseivanlopez Jul 27, 2023
228fc06
WIP: Test volume conversion from D-Bus
joseivanlopez Jul 27, 2023
c57edd2
Update tests for DBus::Storage::Manager and add fixes
ancorgs Aug 8, 2023
f669169
Small modification to the Storage::Manager test
ancorgs Aug 9, 2023
9dd41da
WIP: more tests adaptations and small fixes
ancorgs Aug 10, 2023
02fa693
Rubocop adjustments
ancorgs Aug 14, 2023
5db39de
Fix conversions
joseivanlopez Aug 17, 2023
b28d423
Code style
joseivanlopez Aug 17, 2023
5149822
Relocate tests and minor fixes
joseivanlopez Aug 17, 2023
c042cbc
Fix yardoc
joseivanlopez Aug 17, 2023
389bf34
ToY2Storage tests and some fixes
joseivanlopez Aug 21, 2023
9ede962
More tests and fixes
joseivanlopez Aug 22, 2023
9c3b6cb
Several fixes
joseivanlopez Aug 22, 2023
87d69cf
Tests for volume conversion from Y2Storage
joseivanlopez Aug 23, 2023
eab41d8
Small conversion fix
joseivanlopez Aug 23, 2023
2959194
More conversion tests
joseivanlopez Aug 23, 2023
cb3d031
Rewrite proposal tests
joseivanlopez Aug 23, 2023
dd9b027
Add D-Bus attribute to get the product mount points
joseivanlopez Aug 23, 2023
b1655bc
Add D-Bus tests
joseivanlopez Aug 23, 2023
049f2a9
Avoid unknown space policies
joseivanlopez Aug 23, 2023
a81c12f
Tests for proposal settings reader
joseivanlopez Aug 23, 2023
effeb2a
Some tests for VolumeTemplatsBuilder
ancorgs Aug 24, 2023
6d2ea9a
Add missing mocking
joseivanlopez Aug 24, 2023
b3dfa69
Merge branch 'master' into plane
joseivanlopez Aug 24, 2023
5fcb1cb
Do not memoize volume templates builder
joseivanlopez Aug 24, 2023
a5c37ba
Adapt storage section of config file
joseivanlopez Aug 24, 2023
4f7cd3b
Complete and fix the config file
joseivanlopez Aug 24, 2023
d814b5d
Update D-Bus documentation
joseivanlopez Aug 24, 2023
1d1b5d5
Fix D-Bus method name
joseivanlopez Aug 24, 2023
d0b9db3
doc/dbus: each documented interface goes to a separate .doc.xml file
mvidner Aug 24, 2023
f35db4b
Copy proxy config to the target system
teclator Aug 21, 2023
2e4dcad
Read the current proxy configuration
teclator Aug 21, 2023
1786301
Make install method public
teclator Aug 22, 2023
3a89c89
Added ProxySetup#install unit test
teclator Aug 22, 2023
7ea9c0f
Ensure the read is done out of the chroot
teclator Aug 22, 2023
b92528e
Add the needed packages before installing the software
teclator Aug 23, 2023
bab9875
Move the proxy propose before software proposal is done
teclator Aug 24, 2023
6821ba2
Logger fix and more unit tests
teclator Aug 24, 2023
1b91c24
Enable service in target
teclator Aug 24, 2023
393e021
Added changelog
teclator Aug 28, 2023
f992c17
Merge pull request #711 from openSUSE/proxy_install
teclator Aug 28, 2023
ed12545
weblate-merge-po.yml - fixed PO file validation
lslezak Aug 28, 2023
4ad4cd2
Merge pull request #716 from openSUSE/po_validation_fix
lslezak Aug 28, 2023
82080ec
Update PO files
yast-bot Aug 28, 2023
3e8707e
Fix the connect() function error
imobachgs Aug 11, 2023
6d855b2
Make NetworkSystem generic over the Adapter
imobachgs Aug 11, 2023
ea6cf3b
Allow to inject a different adapter in the NetworkSystem
imobachgs Aug 25, 2023
2835ca5
Write agama-dbus-server integration tests
imobachgs Aug 14, 2023
8f8c5d0
Document a potential bug in the network service
imobachgs Aug 25, 2023
00840d1
Fix NetworkClient to wait for the connection to appear
imobachgs Aug 25, 2023
85ec299
Use a signal to detect when a connection is added
imobachgs Aug 25, 2023
c55edcf
Merge pull request #717 from openSUSE/po_merge_5999537439
lslezak Aug 28, 2023
f22a960
Tests: detect when a service is started
imobachgs Aug 28, 2023
c867656
Run "cargo fmt" happy according to the CI
imobachgs Aug 28, 2023
4dd1ad0
Remove commented (and outdated) code
imobachgs Aug 28, 2023
601135f
Update from code review
imobachgs Aug 28, 2023
e6aff50
Merge pull request #718 from imobachgs/tests
imobachgs Aug 28, 2023
4fa3e39
Mark more texts for translation
lslezak Aug 28, 2023
a7ce2eb
Remove product counter
lslezak Aug 28, 2023
4322aa1
fix wrong name in rake install
jreidinger Aug 28, 2023
6097959
Merge pull request #720 from openSUSE/fix_rake_install
jreidinger Aug 28, 2023
dbad5f9
Merge pull request #719 from openSUSE/add_more_translations
lslezak Aug 28, 2023
67d9a72
Improvements from review
joseivanlopez Aug 28, 2023
cbb6811
Fix some typos
joseivanlopez Aug 29, 2023
7bd1869
Sort requires
joseivanlopez Aug 29, 2023
c9bde61
Make code easier to understand
joseivanlopez Aug 29, 2023
2205da8
Merge branch 'master' into plane
joseivanlopez Aug 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
TargetDevice s
TargetVG s
FsType s
MinSize t
MaxSize t (optinal, max size is considered as unlimited if omitted)
MinSize t (bytes)
MaxSize t (bytes. Optinal, max size is considered as unlimited if omitted)
joseivanlopez marked this conversation as resolved.
Show resolved Hide resolved
AutoSize b
Snapshots b
Outline a{sv}
Expand Down
4 changes: 2 additions & 2 deletions doc/dbus/org.opensuse.Agama.Storage1.Proposal.doc.xml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@
TargetDevice s
TargetVG s
FsType s
MinSize t
MaxSize t (optinal, max size is considered as unlimited if omitted)
MinSize t (bytes)
MaxSize t (bytes. Optinal, max size is considered as unlimited if omitted)
joseivanlopez marked this conversation as resolved.
Show resolved Hide resolved
AutoSize b
Snapshots b
Outline a{sv}
Expand Down
4 changes: 2 additions & 2 deletions service/etc/agama.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ ALP-Dolomite:
- package: device-mapper # Apparently needed if devices at /dev/mapper are used at boot (eg. FDE)
- package: fde-tools # Needed for FDE with TPM, hardcoded here temporarily
archs: aarch64, x86_64
- package: libtss2-tcti-device0 # Same than fde-tools
- package: libtss2-tcti-device0
- package: ppc64-diag # Needed for hardware-based installations
archs: ppc64
optional_packages: null
Expand Down Expand Up @@ -255,7 +255,7 @@ Leap16:
- package: device-mapper # Apparently needed if devices at /dev/mapper are used at boot (eg. FDE)
- package: fde-tools # Needed for FDE with TPM, hardcoded here temporarily
archs: aarch64, x86_64
- package: libtss2-tcti-device0 # Same than fde-tools
- package: libtss2-tcti-device0
optional_packages: null
base_product: Leap16

Expand Down
8 changes: 6 additions & 2 deletions service/lib/agama/dbus/storage/manager.rb
Original file line number Diff line number Diff line change
Expand Up @@ -145,9 +145,13 @@ def default_volume(mount_path)
# @param dbus_settings [Hash]
# @return [Integer] 0 success; 1 error
def calculate_proposal(dbus_settings)
logger.info("Calculating storage proposal from D-Bus settings: #{dbus_settings}")

settings = ProposalSettingsConversion.from_dbus(dbus_settings, config: config)
joseivanlopez marked this conversation as resolved.
Show resolved Hide resolved
logger.info(
"Calculating storage proposal from D-Bus.\n " \
"D-Bus settings: #{dbus_settings}\n" \
"Agama settings: #{settings}"
)

success = proposal.calculate(settings)

success ? 0 : 1
Expand Down
2 changes: 1 addition & 1 deletion service/lib/agama/dbus/storage/proposal.rb
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def initialize(backend, logger)

# Device used as boot device by the storage proposal
#
# @return [String]
# @return [String] Empty string if no device has been selected yet.
def boot_device
dbus_settings.fetch("BootDevice", "")
end
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def initialize(settings)
def convert # rubocop:disable Metrics/AbcSize
{
"BootDevice" => settings.boot_device.to_s,
"LVM" => settings.lvm.enabled,
"LVM" => settings.lvm.enabled?,
"SystemVGDevices" => settings.lvm.system_vg_devices,
"EncryptionPassword" => settings.encryption.password.to_s,
"EncryptionMethod" => settings.encryption.method.id.to_s,
Expand Down
8 changes: 5 additions & 3 deletions service/lib/agama/dbus/storage/volume_conversion/from_dbus.rb
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,10 @@ def target_vg_conversion(target, value)
# @param target [Agama::Storage::Volume]
# @param value [String]
def fs_type_conversion(target, value)
downcase_value = value.downcase

fs_type = target.outline.filesystems.find do |type|
type.to_human_string.downcase == value.downcase
type.to_human_string.downcase == downcase_value
end

return unless fs_type
Expand All @@ -114,13 +116,13 @@ def fs_type_conversion(target, value)
end

# @param target [Agama::Storage::Volume]
# @param value [Integer]
# @param value [Integer] Size in bytes.
def min_size_conversion(target, value)
target.min_size = Y2Storage::DiskSize.new(value)
joseivanlopez marked this conversation as resolved.
Show resolved Hide resolved
end

# @param target [Agama::Storage::Volume]
# @param value [Integer]
# @param value [Integer] Size in bytes.
def max_size_conversion(target, value)
target.max_size = Y2Storage::DiskSize.new(value)
end
Expand Down
4 changes: 3 additions & 1 deletion service/lib/agama/storage/encryption_settings.rb
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,9 @@ class EncryptionSettings
# @return [Y2Storage::EncryptionMethod::Base]
attr_accessor :method

# @return [Y2Storage::PbkdFunction, nil]
# PBKD function to use for LUKS2
#
# @return [Y2Storage::PbkdFunction, nil] Can be nil if using LUKS1.
attr_accessor :pbkd_function

def initialize
Expand Down
6 changes: 1 addition & 5 deletions service/lib/agama/storage/lvm_settings.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ class LvmSettings
#
# @return [Boolean]
attr_accessor :enabled
alias_method :enabled?, :enabled

# Devices to use for the system LVM volume group
#
Expand All @@ -37,11 +38,6 @@ def initialize
@enabled = false
@system_vg_devices = []
end

# @return [Boolean]
def enabled?
!!@enabled
end
end
end
end
3 changes: 2 additions & 1 deletion service/lib/agama/storage/proposal.rb
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,8 @@ def calculate(settings)
# Note that this settings might differ from the {#original_settings}. For example, the sizes
# of some volumes could be adjusted if auto size is set.
#
# @return [ProposalSettings, nil]
# @return [ProposalSettings, nil] nil if no proposal has been calculated yet or the proposal
# was invalidated.
def settings
return nil unless calculated?

Expand Down
2 changes: 1 addition & 1 deletion service/lib/agama/storage/proposal_settings.rb
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ class ProposalSettings
# Device name of the disk that will be used for booting the system and also to allocate all
# the partitions, except those that have been explicitly assigned to other disk(s).
#
# @return [String, nil]
# @return [String, nil] nil if no device has been selected yet.
attr_accessor :boot_device

# Set of volumes to create
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,13 +136,13 @@ def fallbacks_conversion(target)
end
end

# @param mount_path [String]
# @param mount_path [String, nil] nil if not found
def find_min_size_fallback(mount_path)
volume = settings.volumes.find { |v| v.min_size_fallback_for.include?(mount_path) }
volume&.mount_path
end

# @param mount_path [String]
# @param mount_path [String, nil] nil if not found
def find_max_size_fallback(mount_path)
volume = settings.volumes.find { |v| v.max_size_fallback_for.include?(mount_path) }
volume&.mount_path
Expand All @@ -167,7 +167,7 @@ def all_devices
# @return [String, Array<String>]
def device_or_partitions(device)
partitions = partitions(device)
partitions.any? ? partitions : device
partitions.empty? ? device : partitions
end

# @param device [String]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,15 @@ def convert

# @param target [Agama::Storage::Volume]
def sizes_conversion(target)
target.auto_size = !(spec.ignore_fallback_sizes? && spec.ignore_snapshots_sizes?)
target.auto_size = !spec.ignore_fallback_sizes? || !spec.ignore_snapshots_sizes?

# The volume specification contains the min and max sizes for the volume. But the final
# range of sizes used by the Y2Storage proposal depends on the fallback sizes (if this
# volume is fallback for other volume) and the size for snapshots (if snapshots is
# active). The planned device contains the real range of sizes used by the proposal.
#
# From Agama point of view, this is the way of recovering the range of sizes used by
# Y2Storage when a volume is set to have auto size.
planned = planned_device_for(spec.mount_point)
target.min_size = planned&.min || spec.min_size
target.max_size = planned&.max || spec.max_size
joseivanlopez marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
5 changes: 5 additions & 0 deletions service/lib/agama/storage/volume_conversion/to_y2storage.rb
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,11 @@ def sizes_conversion(target)
target.ignore_fallback_sizes = !auto
target.ignore_snapshots_sizes = !auto

# The range of sizes is defined by the volume outline in case of auto size (mix and max
# sizes cannot be configured if auto size is set).
# And note that the inal range of sizes used by the Y2Storage proposal is calculated by
joseivanlopez marked this conversation as resolved.
Show resolved Hide resolved
# Y2Storage according the range configured here and other sizes like fallback sizes or
# the size for snapshots.
target.min_size = auto ? volume.outline.base_min_size : volume.min_size
target.max_size = auto ? volume.outline.base_max_size : volume.max_size
joseivanlopez marked this conversation as resolved.
Show resolved Hide resolved
end
Expand Down
24 changes: 18 additions & 6 deletions service/lib/agama/storage/volume_outline.rb
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,13 @@ class VolumeOutline
attr_accessor :filesystems

# Base value to calculate the min size for the volume (if #auto_size is set to true
# for that final volume) or to use as default value (if #auto_size is false)
# for that final volume)
#
# @return [Y2Storage::DiskSize]
attr_accessor :base_min_size

# Base value to calculate the max size for the volume (if #auto_size is set to true
# for that final volume) or to use as default value (if #auto_size is false)
# for that final volume)
#
# @return [Y2Storage::DiskSize]
attr_accessor :base_max_size
Expand All @@ -56,10 +56,22 @@ class VolumeOutline
attr_accessor :adjust_by_ram
alias_method :adjust_by_ram?, :adjust_by_ram

# @return [Array<String>] mount paths of other volumes
# Lists the mount paths of the volumes for which this volume is a min size fallback.
#
# Being a min size fallback means that the min size of the volume would be increased by the
# min size of other volumes, if any of that other volumes is not used for the proposal.
#
# For example, let's say the root volume is a fallback for the min size of /home and /var
# (root.min_size_fallback_for => ["/home", "/var"]). And a proposal is calculated with only
# root and /home. In that case, the min size of /var is added to the min size of root. The
# same would happen for /home if the proposal does not include it.
joseivanlopez marked this conversation as resolved.
Show resolved Hide resolved
#
# @return [Array<String>]
attr_accessor :min_size_fallback_for

# @return [Array<String>] mount paths of other volumes
# The same as {#min_size_fallback_for}, but for the max size of the volume.
#
# @return [Array<String>]
attr_accessor :max_size_fallback_for

# Whether snapshots option can be configured
Expand All @@ -70,12 +82,12 @@ class VolumeOutline

# Size required for snapshots
#
# @return [Y2Storage::DiskSize, nil]
# @return [Y2Storage::DiskSize, nil] nil if no extra size for snapshots.
attr_accessor :snapshots_size

# Percentage of space required for snapshots
#
# @return [Integer, nil]
# @return [Integer, nil] nil if no extra size for snapshots.
attr_accessor :snapshots_percentage

def initialize
Expand Down
Loading