Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Final arm64 related functionality #1087
Conversation
|
The commit message on commit af4d5a0 looks incomplete. |
| guest_mac: None, | ||
| rx_rate_limiter: None, | ||
| tx_rate_limiter: None, | ||
| allow_mmds_requests: false, | ||
| tap: None, | ||
| }; | ||
| assert!(vmm.insert_net_device(network_interface).is_ok()); | ||
| vmm.insert_net_device(network_interface).expect("lala"); |
petreeftime
May 9, 2019
Contributor
The expect message is not very helpful.
The expect message is not very helpful.
dianpopa
May 14, 2019
Author
Contributor
Sorry, some leftover debug messages.
Sorry, some leftover debug messages.
|
|
||
| self.configure_system()?; | ||
| #[cfg(target_arch = "aarch64")] |
petreeftime
May 9, 2019
Contributor
Maybe it's explained in the missing part of the commit message, but it's not clear why these two setup steps are so different. Is it an error on x86_64 to create the vcpus earlier? Why can't the first step be load_kernel on both?
Maybe it's explained in the missing part of the commit message, but it's not clear why these two setup steps are so different. Is it an error on x86_64 to create the vcpus earlier? Why can't the first step be load_kernel on both?
dianpopa
May 14, 2019
Author
Contributor
I updated the commit message and also put a comment before branching the implementation. Take a look.
I updated the commit message and also put a comment before branching the implementation. Take a look.
petreeftime
May 14, 2019
Contributor
Looks much cleaner now.
Could it be possible to further reduce duplication by doing the following:
let entry_addr = self.load_kernel()?;
#[cfg(target_arch = "aarch64")]
vcpus = self.create_vcpus(entry_addr, request_ts)?;
self.setup_interrupt_controller()?;
self.attach_virtio_devices()?;
self.attach_legacy_devices()?;
#[cfg(target_arch = "x86_64")]
vcpus = self.create_vcpus(entry_addr, request_ts)?;
Looks much cleaner now.
Could it be possible to further reduce duplication by doing the following:
let entry_addr = self.load_kernel()?;
#[cfg(target_arch = "aarch64")]
vcpus = self.create_vcpus(entry_addr, request_ts)?;
self.setup_interrupt_controller()?;
self.attach_virtio_devices()?;
self.attach_legacy_devices()?;
#[cfg(target_arch = "x86_64")]
vcpus = self.create_vcpus(entry_addr, request_ts)?;
acatangiu
May 15, 2019
Contributor
nit: Would be nice to also add a comment for both like:
// On aarch64 we need to create vcpus before creating the interrupt controller.
#[cfg(target_arch = "aarch64")]
vcpus = self.create_vcpus(entry_addr, request_ts)?;
...
// On x86_64 we need to create vcpus after setting up everything else.
#[cfg(target_arch = "x86_64")]
vcpus = self.create_vcpus(entry_addr, request_ts)?;
(Or smth similar)
nit: Would be nice to also add a comment for both like:
// On aarch64 we need to create vcpus before creating the interrupt controller.
#[cfg(target_arch = "aarch64")]
vcpus = self.create_vcpus(entry_addr, request_ts)?;
...
// On x86_64 we need to create vcpus after setting up everything else.
#[cfg(target_arch = "x86_64")]
vcpus = self.create_vcpus(entry_addr, request_ts)?;
(Or smth similar)
| self.attach_virtio_devices()?; | ||
| self.attach_legacy_devices()?; | ||
|
|
||
| self.register_events()?; |
petreeftime
May 9, 2019
Contributor
Why are register_events and configure_system swapped?
Why are register_events and configure_system swapped?
dianpopa
May 14, 2019
Author
Contributor
No reason and I also just realized that there is no reason in having them inside the aarch64 cfg. I removed them from there. Take a look.
No reason and I also just realized that there is no reason in having them inside the aarch64 cfg. I removed them from there. Take a look.
|
Really nice structure and separation of ARM and X86_64, cool! |
| @@ -272,7 +283,7 @@ mod tests { | |||
| let intr_evt = EventFd::new().unwrap(); | |||
| let serial_out = SharedBuffer::new(); | |||
|
|
|||
| let mut serial = Serial::new_out(intr_evt, Box::new(serial_out.clone())); | |||
| let mut serial = Serial::new_out(intr_evt, Box::new(serial_out.clone()), None); | |||
acatangiu
May 10, 2019
Contributor
Is this test not run on aarch64? Wouldn't it need len=4 on arm?
Same question for all following tests.
Is this test not run on aarch64? Wouldn't it need len=4 on arm?
Same question for all following tests.
dianpopa
May 14, 2019
Author
Contributor
It is run on aarch64 too, but given that this is an unit test it will hit the same code like on x86.
What makes a difference here is the enable_earlycon (from a later commit) function that indeed is specific to aarch64 (not even specific it is just that on aarch64 we need to do it through MMIO addreses os the data.len will be 4).
TO be honest, what I see worth unit testing here is checking that the METRICS.uart.missed_read_count.inc();
stays 0 after a read that has the same length as the length to what the device was instructed to accept inside the constructor.
Another thing that we could test here is that when we initiliaze the length to some different length different than 1, the device knows to read data only of that length.
It is run on aarch64 too, but given that this is an unit test it will hit the same code like on x86.
What makes a difference here is the enable_earlycon (from a later commit) function that indeed is specific to aarch64 (not even specific it is just that on aarch64 we need to do it through MMIO addreses os the data.len will be 4).
TO be honest, what I see worth unit testing here is checking that the METRICS.uart.missed_read_count.inc();
stays 0 after a read that has the same length as the length to what the device was instructed to accept inside the constructor.
Another thing that we could test here is that when we initiliaze the length to some different length different than 1, the device knows to read data only of that length.
dianpopa
May 14, 2019
Author
Contributor
Exercised all of the above with test_serial_data_len. Take a look.
Exercised all of the above with test_serial_data_len. Take a look.
| addr: u64, | ||
| irq: u32, | ||
| len: u64, | ||
| type_: u32, |
acatangiu
May 10, 2019
Contributor
nit: Maybe dev_type instead of type_?
nit: Maybe dev_type instead of type_?
dianpopa
May 14, 2019
Author
Contributor
I thought about this but dev_type inside MMIODeviceInfo?
I thought about this but dev_type inside MMIODeviceInfo?
acatangiu
May 15, 2019
Contributor
I thought about this but dev_type inside MMIODeviceInfo?
Sounds good to me 😆 , but like I said, it's a nit.
I thought about this but dev_type inside MMIODeviceInfo?
Sounds good to me
| #[derive(Clone, Debug)] | ||
| pub struct MMIODeviceInfo { | ||
| addr: u64, | ||
| irq: u32, |
acatangiu
May 10, 2019
Contributor
nit: reorder suggestion: put len right after addr since it's a familiar pattern.
nit: reorder suggestion: put len right after addr since it's a familiar pattern.
dianpopa
May 14, 2019
Author
Contributor
I tried to maintain alphabetical order :(
I tried to maintain alphabetical order :(
acatangiu
May 15, 2019
Contributor
Still a nit, but addr and len are logically tied, they describe the device address space slice, and that trumps alphabetical order for me.
Still a nit, but addr and len are logically tied, they describe the device address space slice, and that trumps alphabetical order for me.
|
|
||
| if let Some(device_id) = id { | ||
| self.id_to_addr_map.insert(device_id.clone(), ret); | ||
| self.id_to_dev_info.insert( | ||
| device_id.clone(), |
acatangiu
May 10, 2019
Contributor
Doesn't need cloning. You're already cloning when calling this function. Either:
- make
id param as id: Option<&String> and clone it here (callers of the func would not clone id anymore),
or
- leave
id param as it is, and consume it here instead of cloning.
Doesn't need cloning. You're already cloning when calling this function. Either:
- make
idparam asid: Option<&String>and clone it here (callers of the func would not cloneidanymore),
or - leave
idparam as it is, and consume it here instead of cloning.
dianpopa
May 14, 2019
Author
Contributor
Done, I actually made id mandatory (not an Option anymore). Leaving it as Option made the code very prone to programming errors since on aarch64, non mentioning an id would not make it part of the FDT. Take a look.
Done, I actually made id mandatory (not an Option anymore). Leaving it as Option made the code very prone to programming errors since on aarch64, non mentioning an id would not make it part of the FDT. Take a look.
| fn addr(&self) -> u64; | ||
| fn irq(&self) -> u32; | ||
| fn length(&self) -> u64; | ||
| fn type_(&self) -> u32; |
acatangiu
May 10, 2019
Contributor
type_ would look better as an enum clearly stating virtio or serial instead of an u32 on which we map certain numbers to certain types.
type_ would look better as an enum clearly stating virtio or serial instead of an u32 on which we map certain numbers to certain types.
dianpopa
May 14, 2019
Author
Contributor
Done, take a look.
Done, take a look.
| .unwrap(); | ||
| output.write_all(&dtb).unwrap(); | ||
| } | ||
| { |
acatangiu
May 10, 2019
Contributor
nit: looks like some extra indentation slipped in here
nit: looks like some extra indentation slipped in here
dianpopa
May 14, 2019
Author
Contributor
Done.
Done.
| #[cfg(target_arch = "aarch64")] | ||
| /// Gets the information of the devices registered up to some point in time. | ||
| pub fn get_device_info(&self) -> HashMap<String, MMIODeviceInfo> { | ||
| self.id_to_dev_info.clone() |
acatangiu
May 10, 2019
Contributor
Haven't looked at the implications, but would like to see this returning a reference instead of a clone.
Haven't looked at the implications, but would like to see this returning a reference instead of a clone.
dianpopa
May 14, 2019
Author
Contributor
I can indeed return a reference here. Take a look.
I can indeed return a reference here. Take a look.
| @@ -1162,17 +1162,9 @@ impl Vmm { | |||
| .ok_or(StartMicrovmError::VcpusNotConfigured)?; | |||
acatangiu
May 10, 2019
Contributor
Commit af4d5a0b292dfc6ef6b381d75f76c4f5ff2c3701 has an incomplete description.
Commit af4d5a0b292dfc6ef6b381d75f76c4f5ff2c3701 has an incomplete description.
dianpopa
May 14, 2019
Author
Contributor
Solved.
Solved.
| guest_mac: None, | ||
| rx_rate_limiter: None, | ||
| tx_rate_limiter: None, | ||
| allow_mmds_requests: false, | ||
| tap: None, | ||
| }; | ||
| assert!(vmm.insert_net_device(network_interface).is_ok()); | ||
| vmm.insert_net_device(network_interface).expect("lala"); | ||
| //assert!(vmm.insert_net_device(network_interface).is_ok()); |
acatangiu
May 10, 2019
Contributor
Leftover commented line.
Leftover commented line.
dianpopa
May 14, 2019
Author
Contributor
Solved.
Solved.
|
LGTM. |
| fn length(&self) -> u64 { | ||
| self.len | ||
| } | ||
| fn type_(&self) -> FDTDeviceType { |
acatangiu
May 15, 2019
Contributor
What do you think about making type_ in MMIODeviceInfo also an enum, ideally the same enum as FDTDeviceType.
I think you can call this something generic (not fdt related) and define it for both x86_64 and aarch64 in order to use it everywhere.
What do you think about making type_ in MMIODeviceInfo also an enum, ideally the same enum as FDTDeviceType.
I think you can call this something generic (not fdt related) and define it for both x86_64 and aarch64 in order to use it everywhere.
dianpopa
May 16, 2019
Author
Contributor
Good suggestion. Take a look.
Good suggestion. Take a look.
The register_device function inside the mmio device manager actually adds a virtio device. You can also have mmio devices added to the bus without them actually using virtio protocol. Signed-off-by: Diana Popa <dpopa@amazon.com>
By default our console implementation is hardcoded to read 1 byte since we expect the data to come from a I/O port. However, when implementing early console support through MMIO addresses, the length of bytes received at that address is 4. Signed-off-by: Diana Popa <dpopa@amazon.com>
We need it for loading the kernel for each architecture. Signed-off-by: Diana Popa <dpopa@amazon.com>
CMDLINE_MAX_ADDR was removed from common interface since loading the commandline to a specific address is x86_64 specific. Signed-off-by: Diana Popa <dpopa@amazon.com>
Signed-off-by: Diana Popa <dpopa@amazon.com>
Up until now we could only obtain the address where some mmio device had been registered. For creating the Flattened Device Tree we also need the length and the irq associated with it. Signed-off-by: Diana Popa <dpopa@amazon.com>
aarch64 has no I/O port support so we cannot enable serial the same way we do for x86_64. We are enabling it by adding an earlycon device at some MMIO address. Signed-off-by: Diana Popa <dpopa@amazon.com>
Signed-off-by: Diana Popa <dpopa@amazon.com>
On aarch64, KVM_CREATE_VCPUS needs to be called before setting up the interrupt controller (i.e GIC) so the sequence of start operations was adjusted to fit kernel requirements. Signed-off-by: Diana Popa <dpopa@amazon.com>
By default, on aarch64 the rootfs is mounted in readonly mode if not specified otherwise. Signed-off-by: Diana Popa <dpopa@amazon.com>
Signed-off-by: Diana Popa <dpopa@amazon.com>
Signed-off-by: Diana Popa <dpopa@amazon.com>
|
|
10fdf2d
into
firecracker-microvm:master
With this functionality, I can ssh into an aarch64 guest.
Replication steps to come.
Still some bugs need to be solved:
Issue #, if available: Fixes #757.
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.