Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Qemu Ports Not Mapping #10033

Open
tschmi5 opened this issue Feb 16, 2021 · 4 comments · May be fixed by #10035
Open

Qemu Ports Not Mapping #10033

tschmi5 opened this issue Feb 16, 2021 · 4 comments · May be fixed by #10035
Labels
theme/driver/qemu type/bug type/enhancement

Comments

@tschmi5
Copy link

tschmi5 commented Feb 16, 2021

Nomad Version 1.0.3

I am using a Nomad 1.0.3/ Consul 1.9.3/ Centos 7 Running within VirtualBox VMs.

Running Nomad & Consul as root

It appears as if the networking/port configuration is not working correctly for the Qemu Driver. I have tried all the port configurations and none are exposing the VM port (8080) I have. I have verified through several methods that the VM is not the issue and port 8080 is exposed on all clients.

HCL Config

job "javaVM" {
  datacenters = ["dc1"]

  group "vms" {
    network {  
      // I have also tried dynamic here to no avail
      port "http" {
        static = 8080
      }
    }

    ephemeral_disk {
      size = 1024
    }

    task "petclinicvm" {
      # The "driver" parameter specifies the task driver that should be used to
      # run the task.
      driver = "qemu"

      config {
        image_path = "local/java-petclinic.qcow2"
        port_map {
      	  http = 8080
        }
        // I have also tried directly setting it but this causes the task to fail
        // args = [
        //  "-net user,hostfwd=tcp::8080-:8080",
        //  "-net nic"
        // ]
      }
      artifact {
        source = "http://10.0.0.200:8000/JavaVM/java-petclinic.qcow2"
	      destination = "local"
      }
      resources {
        cpu    = 1000 
        memory = 2048 
      }
    }
  }
}

Job Status

ID                  = bd012930-556f-10ad-dbab-fc242223d4fb
Eval ID             = 7abdd977
Name                = javaVM.vms[0]
Node ID             = 62049756
Node Name           = client2
Job ID              = javaVM
Job Version         = 0
Client Status       = running
Client Description  = Tasks are running
Desired Status      = run
Desired Description = <none>
Created             = 38s ago
Modified            = 20s ago
Deployment ID       = b561a57c
Deployment Health   = healthy

Allocation Addresses
Label  Dynamic  Address
*http  yes      10.0.0.76:8080

Task "petclinicvm" is "running"
Task Resources
CPU            Memory           Disk     Addresses
4168/1000 MHz  826 MiB/2.0 GiB  1.0 GiB  

Task Events:
Started At     = 2021-02-16T14:49:59Z
Finished At    = N/A
Total Restarts = 0
Last Restart   = N/A

Recent Events:
Time                       Type                   Description
2021-02-16T09:49:59-05:00  Started                Task started by client
2021-02-16T09:49:52-05:00  Downloading Artifacts  Client is downloading artifacts
2021-02-16T09:49:52-05:00  Task Setup             Building Task Directory
2021-02-16T09:49:52-05:00  Received               Task received by client

Ports on host where VM is running

[root@client2 ~]# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1784/dnsmasq        
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1074/sshd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1296/master         
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      687/rpcbind         
tcp6       0      0 :::8500                 :::*                    LISTEN      1059/consul         
tcp6       0      0 :::8502                 :::*                    LISTEN      1059/consul         
tcp6       0      0 :::22                   :::*                    LISTEN      1074/sshd           
tcp6       0      0 :::8600                 :::*                    LISTEN      1059/consul         
tcp6       0      0 ::1:25                  :::*                    LISTEN      1296/master         
tcp6       0      0 :::4646                 :::*                    LISTEN      1065/nomad          
tcp6       0      0 :::8301                 :::*                    LISTEN      1059/consul         
tcp6       0      0 :::111                  :::*                    LISTEN      687/rpcbind         
udp        0      0 192.168.122.1:53        0.0.0.0:*                           1784/dnsmasq        
udp        0      0 0.0.0.0:67              0.0.0.0:*                           1784/dnsmasq        
udp        0      0 0.0.0.0:68              0.0.0.0:*                           869/dhclient        
udp        0      0 0.0.0.0:111             0.0.0.0:*                           687/rpcbind         
udp        0      0 0.0.0.0:53437           0.0.0.0:*                           5892/qemu-kvm       
udp        0      0 0.0.0.0:39165           0.0.0.0:*                           5892/qemu-kvm       
udp        0      0 127.0.0.1:323           0.0.0.0:*                           707/chronyd         
udp        0      0 0.0.0.0:861             0.0.0.0:*                           687/rpcbind         
udp        0      0 0.0.0.0:48355           0.0.0.0:*                           5892/qemu-kvm       
udp        0      0 0.0.0.0:54510           0.0.0.0:*                           5892/qemu-kvm       
udp        0      0 0.0.0.0:54684           0.0.0.0:*                           5892/qemu-kvm       
udp        0      0 0.0.0.0:48554           0.0.0.0:*                           5892/qemu-kvm       
udp        0      0 0.0.0.0:60862           0.0.0.0:*                           5892/qemu-kvm       
udp        0      0 0.0.0.0:48886           0.0.0.0:*                           5892/qemu-kvm       
udp6       0      0 :::8301                 :::*                                1059/consul         
udp6       0      0 :::111                  :::*                                687/rpcbind         
udp6       0      0 ::1:323                 :::*                                707/chronyd         
udp6       0      0 :::8600                 :::*                                1059/consul         
udp6       0      0 :::861                  :::*                                687/rpcbind

I found this in the Interweb Ethers so Im not sure if this truly means much but it appears no ports configs are being passed to the Qemu command when run

[root@client2 ~]# ps -ef | awk -e '/qemu/ && !/awk/' | sed -e 's/[^/]*//' -e 's/ -/\n\t-/g'
/usr/libexec/qemu-kvm
	-machine type=pc,accel=tcg
	-name java-petclinic.qcow2
	-m 2048M
	-drive file=local/java-petclinic.qcow2
	-nographic

The last anecdote I would like to add is that when I setup qemu the command for the binary is 'qemu-kvm' and I just created a link within the path named 'qemu-system-x86_64'. As far as I can tell it seems to be the same thing but renamed.

Am I missing something?
@tgross
Copy link
Member

tgross commented Feb 16, 2021

Hi @tschmi5! You're not missing anything... we got network isolation working for QEMU recently #9860 but we still need to get the port mapping fixed. We have a draft PR open at #10035 but we need to get that re-assigned to finish it up.

@tgross tgross linked a pull request Feb 16, 2021 that will close this issue
WIP further updates to the QEMU driver. #10035
Draft
@carmelo42
Copy link

Hello ! Any news on this issue ? I can't use any qemu VM with nomad if I can't access it :(

@tgross tgross mentioned this issue May 2, 2022
Closed
@tgross tgross added the type/bug label May 2, 2022
@iovcho
Copy link

iovcho commented Jul 20, 2022

Hello I have the same issue. port_map not working with qemu driver.

@jhillyerd
Copy link

Why is qemu config.port_map in the docs if it is not implemented? This is misleading.

@tgross tgross moved this to Needs Roadmapping in Nomad - Community Issues Triage Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
theme/driver/qemu type/bug type/enhancement
Projects
Nomad - Community Issues Triage
Status: Needs Roadmapping
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

WIP further updates to the QEMU driver. hashicorp/nomad
5 participants
@tgross @carmelo42 @jhillyerd @iovcho @tschmi5