Incredibly slow disk write/read times with `.macvm`

We run macos arm `.macvm` bundles on a cluster of ec2 `mac2.metal` instances. We've noticed Parallels macvms have incredibly slow disk read/write on these hosts. This is causing items like automated xcode installations to take over 45 minutes.

No matter what configuration we try, we get disk write about this speed:
```
write=$(dd if=/dev/zero bs=2048k of=tstfile count=1024 2>&1 | grep sec | awk '{print $1 / 1024 / 1024 / $5, "MB/sec" }')
echo $write
32.686 MB/sec
```

```
time sh -c "dd if=/dev/zero of=test.tmp bs=4k count=524288 && sync"

524288+0 records in

524288+0 records out

2147483648 bytes transferred in 53.625618 secs (40045854 bytes/sec)
```

We've tried items like creating an empty sparse disk image using this guide which results in same performance, moving where we store/run vms, etc. Nothing seems to help at all.

 

Looking through logs, it's using the wrong host disk. We utilize a fresh brand new secondary volume to run these vms (which they are stored in). How can we instruct desktop to use a specific disk/volume?

 

I was able to resolve this (for the secondary attached disk image).

It looks like it's being mounted incorrectly. Once I erased/formatted it after starting the vm, we had 450mb/s.

 

This one is on me actually. I should have set `-format` to `UDRW` when creating the disk image.

 

Appears I spoke too soon again. There is some kind of weird potential fragmentation going on or something. After 1-2 writes, disk writes go back to ~30mb/s.

 

I now believe this is actually the virtual disk you all create for the `.macvm`.

I receive very low write speeds for the `disk0.img` and my secondary volume. The only way it runs fast is if I make it `hfs`. Unfortunately, our users have requirements for `APFS` so I'm not sure how to diagnose this.

Since both are considered internal disks for the vm, could this be some kind of cause?

 

@Avinash Bundhoo what should be expected write speed for m1 with an allocated 6gb mem and 2cpu? Has anyone else reported intermittent write spikes like this? Been trying to debug for this a month now. Not sure why HFS is seamless as a secondary virtual disk image but APFS is sporadic at best.

 

Here is a better view of the write jumps. It randomly goes up to ~300mb/s and then immediately back down to ~30mb/s. I've tried tweaking everything from spotlight indexing, various `apfs` container setups, purging cache, etc, nothing has provided any kind of consistency or been reproducible.

 

This is a pretty major block for us. Any idea or update @Avinash Bundhoo ?