Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How does lume differ from lima? #10

Open
changhoon-sung opened this issue Feb 4, 2025 · 13 comments
Open

How does lume differ from lima? #10

changhoon-sung opened this issue Feb 4, 2025 · 13 comments
Labels
discussion Discussions about the project

Comments

@changhoon-sung
Copy link

We can see that there are quite a few macOS vz wrappers available. Some, like UTM, orbstack.dev, and VirtualBuddy, offer both CLI and GUI interfaces, while Lima stands out as a CLI tool with a strong community.

What advantages does lume have over these existing solutions? Explaining these differences would make your project more compelling.

@changhoon-sung changhoon-sung changed the title How Does lume Differ from lima? How does lume differ from lima? Feb 4, 2025
@gvkhna
Copy link

gvkhna commented Feb 4, 2025

right off the bat i see this is swift and lima is golang. Not sure of any real pro/con, but a swift project could be quite attractive if it better ties into the native api’s. Starred.

@changhoon-sung
Copy link
Author

@gvkhna VirtualBuddy is 97% Swift

@gromgit
Copy link

gromgit commented Feb 4, 2025

I took interest in lume because macOS VMs. I don't think lima does that.

@changhoon-sung
Copy link
Author

@gromgit Good point, lima is targeting Linux from its name, so maybe macos vms are not in their interest. But VirtualBuddy and UTM both support macOS VM. My counterpoint may sound aggressive and skeptical, but it's pure curiosity about people redeveloping tools that already exist.

@gvkhna
Copy link

gvkhna commented Feb 4, 2025

I actually couldn’t get UTM to work last time i tried it, had to switch to qemu directly. This seems like an attempt at a more native lima/colima setup. I’m all for it, colima works fine but it’s still clearly very early.

@alessandropuppo
Copy link

alessandropuppo commented Feb 4, 2025

The main differences between Lume and Lima are that:

  • Lima focuses on Linux VMs and doesn't support managing macOS VMs, while Lume does.

  • Lima is more of a container-oriented way of spinning up Linux VMs. We're still debating whether to go in that direction with Lume, but the good news is that it would mostly require tweaking the hooks to expose Lume’s core to adopt the containerd standard. I’d love to hear your thoughts - would you find it useful to have a Docker-like interface for starting macOS workloads? Similar to: https://github.com/qemus/qemu-docker

  • There are still many dependencies on QEMU, which doesn't play well with Apple silicon - while we opted to support only M chips (80-90% of the market cap today) by relying on the latest Apple Virtualization.Framework bits.

@f-trycua
Copy link
Contributor

f-trycua commented Feb 4, 2025

I appreciate @changhoon-sung @gvkhna @gromgit for starting this thread. What @sandrostar mentioned is spot on, and I’d like to add some more context around the reasoning behind this project:

We created lume to meet our own bar for developing new OSS tools on Apple Silicon, which we’ll be announcing in the next few weeks. These tools will build on top of lume and pylume. While working on the north-star experience, we saw an opportunity to package the core solution as a CLI and local API server, and open-source it.

We took that step, and the unexpected success of posting it on HN was a pleasant surprise.

To clarify where we see lume today - and the direction we believe the support from the open-source community can take it moving forward:

  • macOS-first VM Support: Unlike lima, which is primarily focused on Linux, lume is designed with macOS-first VM support. It might not be obvious now, but this will be more evident in the upcoming releases, where the feature set for macOS VMs will expand.
  • Extensibility: Many existing vz tools lacked support for VNC sessions, remote VM access, unattended setup, and pulling from remote ghcr registries. These gaps led us to create lume. We aim for a 1-click warmed-up run, enabling users to quickly get VMs up and running without long setup times. Additionally, we recognized scenarios where a command line interface alone was not enough to meet our objectives, which is why we exposed the core functionalities through a local server and used that for pylume, a Pythonic library for developers to complement lume.
  • Licensing: We also wanted to offer a more flexible license than some open-source solutions targeting macOS, which often have restrictive licenses. lume is released under the MIT license and will always remain so, encouraging community contributions.

UI Automation and Sandboxing are two key areas we’re focused on next. With the recent wave of general AI agents, there’s an increasing need for isolated environments to run experiments, or even for AI agents to interact with computer interfaces. Interestingly, we’ve already seen some comments on HN pointing out use cases around this, and we’re thrilled to share more soon.

I’m happy to keep the issue open to discuss lume’s direction from here.

@changhoon-sung
Copy link
Author

changhoon-sung commented Feb 5, 2025

@sandrostar @f-trycua Thank you for providing more context. I also believe that the local API server offered by lume is a key differentiator from other projects. Additionally, understanding this project as a foundation for future products rather than a standalone product makes more senses. I’m looking forward to the upcoming releases—wishing you all the best.

I have a question and following sub questions:

What are the use cases for leveraging macOS VMs?

  • Is the goal to provide application services in an isolated environment, similar to containers?
  • If so, is the main problem that macOS VMs aim to solve performance-related issues, such as the overhead of Linux VMs plus UX/UI?
  • In this case, macOS VMs would not be able to use Linux’s cgroup and namespace-based process isolation mechanisms (container in short) unless it runs nested Linux VM. What are your thoughts on this trade-off?
  • Or, are there workloads requiring OS-level isolation beyond application-level?

@jedisct1
Copy link

jedisct1 commented Feb 6, 2025

Great work!

That would require an intermediary VM, but are there any plans to support running x86_64 Linux VMs via Rosetta?

Doing this with Lima is possible, but painful. A tool to do that with a simple command would be awesome.

@f-trycua
Copy link
Contributor

f-trycua commented Feb 6, 2025

Sorry for the delay; GitHub was down in my region for most of the day yesterday! Let me address them one by one.

Is the goal to provide application services in an isolated environment, similar to containers?

Yes. Some potential use cases requiring an isolated Darwin environment for application-level services (as suggested by HN users in the comments on our post):

  • GUI Applications: macOS VMs enable isolated environments for testing and running native macOS apps, including GUI-based apps like Xcode, which containers can't support.
  • Darwin-Specific Development: Testing apps built with frameworks like SwiftUI or AppKit, ensuring compatibility across different macOS versions or managing multiple environments with different versions of macOS.
  • CI/CD and QA Testing: Isolated macOS VMs are useful for continuous integration, testing, and sandboxed environments that are easy to reset and manage. This is particularly valuable for teams needing specific versions of Xcode or macOS.
  • UI Automation: macOS VMs allow automation for interacting with macOS GUIs, useful for testing, task automation, or computer-use AI interactions with the OS.
  • Security and Isolation: Running potentially risky applications, such as experimenting with LLM models or testing malicious code, in a controlled environment.

For typical workloads, particularly CPU-related tasks, using lume with Apple’s Virtualization.framework won't lead to noticeable performance issues. However, for more GPU-related workloads (like running LLM models), there's currently no GPU pass-through support within the Virtualization.framework. To improve this, we could possibly leverage Apple's Hypervisor.framework (i.e. KVM ioctl equivalent), similar to how libkrun enables GPU-passthrough for Linux VMs.

If so, is the main problem that macOS VMs aim to solve performance-related issues, such as the overhead of Linux VMs plus UX/UI?

Yes, to an extent, plus Darwin application compatibility.

In this case, macOS VMs would not be able to use Linux’s cgroup and namespace-based process isolation mechanisms (container in short) unless it runs nested Linux VM. What are your thoughts on this trade-off?

I guess it's a trade-off based on different needs.

  • Container Workloads: For workloads requiring Linux's cgroup and namespace-based isolation, you could implement a Docker-compatible interface for macOS VMs (e.g., similar to qemu-docker and dockur) alongside a sidecar Linux container, e.g. with Podman and libkrun. This could be an interesting direction for another interface based on lume.
  • Nested VMs: Running nested Linux VMs is another option but comes with perf considerations.

Or, are there workloads requiring OS-level isolation beyond application-level?

Not that I can think of, but I haven’t fully wrapped my mind around it yet. It would be interesting to hear your thoughts.

@f-trycua
Copy link
Contributor

f-trycua commented Feb 6, 2025

Great work!

That would require an intermediary VM, but are there any plans to support running x86_64 Linux VMs via Rosetta?

Doing this with Lima is possible, but painful. A tool to do that with a simple command would be awesome.

Hi @jedisct1 - although not (yet) documented, it should be supported with lume today: https://github.com/trycua/lume/blob/main/src/Virtualization/VMVirtualizationService.swift#L292

We haven't published any x86_64 images in the ghcr registry yet, so you would need to follow this guide to create a custom VM: https://github.com/trycua/lume/blob/main/docs/FAQ.md#how-do-i-install-a-custom-linux-image. Once prepared, you can clone it with lume clone <vm_name> <new_vm_name> and reuse it as a gold image.

I'm curious - what x86_64 distro would you like to see as part of our prebuilt images?

@jedisct1
Copy link

jedisct1 commented Feb 6, 2025

Awesome!

I'm curious - what x86_64 distro would you like to see as part of our prebuilt images?

Ubuntu-LTS (same as the current aarch64 image) and Alpine.

@changhoon-sung
Copy link
Author

Not that I can think of, but I haven’t fully wrapped my mind around it yet. It would be interesting to hear your thoughts.

I largely agree with your perspective. The majority of VM-based use cases fall under application isolation, as you mentioned, and the lack of an official container technology on macOS remains a limitation at this point.

I also agree that VM usage can help address certain OS-dependent tasks, such as GUI and device testing, which require specific macOS versions.

While I don’t personally have Darwin-dependent workloads, I acknowledge the challenges in this gap. I hope that Lume and the upcoming solutions can bridge this transitional period (I believe and hope this is a transitional period for macOS) and become valuable projects in this domain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion Discussions about the project
Projects
None yet
Development

No branches or pull requests

6 participants