Editing XigmaNAS (section)

=== VM Setup ===
https://github.com/churchers/vm-bhyve

Type ~+Ctrl-D to exit back to your host.
<pre>
# sysrc vm_enable="YES"
# sysrc vm_dir="zfs:dozer/vms"
# zfs create -o compression=lz4 dozer/vms/zvols # lz4 is better for multiple reads?
vm init
vm img https://download.freebsd.org/ftp/releases/VM-IMAGES/14.1-RELEASE/amd64/Latest/FreeBSD-14.1-RELEASE-amd64-BASIC-CLOUDINIT-zfs.raw.xz 
# zfs create -s -V 35G dozer/vms/zvols/freebsd141
# dd bs=1m if=~/vms/.img/FreeBSD-14.1-RELEASE-amd64-BASIC-CLOUDINIT-zfs.raw of=/dev/zvol/dozer/vms/zvols/freebsd141 status=progress
# zfs snapshot dozer/vms/zvols/freebsd141@img35G
#vm create -t xbuild -s 35G -i FreeBSD-14.1-RELEASE-amd64-BASIC-CLOUDINIT-zfs.raw -C -k ~/.ssh/vms_id_ed.pub xBhyve141
vm create -t freebsd-zvol -C -k ~/.ssh/vms_id_ed.pub ci
zfs destroy dozer/vms/ci/disk0
zfs clone dozer/vms/zvols/freebsd141@img35G dozer/vms/ci/disk0

 gpart recover nda0
 gpart show
# gpart resize -i 5 nda0
# growfs -f /dev/gpt/rootfs

vm start xBhyve141
vm console xBhyve141
</pre>
graphics is only possible with loader="uefi", not with "grub". https://forums.freebsd.org/threads/bhyve-graphics-install-and-vncviewer.88114/
disk0.img is faster, but doesn't support -s 35G on create
nvme is faster than virtio-blk
<pre>
vm create -t freebsd-zvol -s 35G -i FreeBSD-14.1-RELEASE-amd64-BASIC-CLOUDINIT-zfs.raw -C -k ~/.ssh/vms_id_ed.pub xBuild141
dd if=/dev/zvol/dozer/vms/xBuild141/disk0 of=xBuild141/disk0.img bs=1m
vm configure xBuild141
disk0_name="disk0.img"
#disk0_dev="sparse-zvol"

vm create -t freebsd -s 35G -i FreeBSD-14.1-RELEASE-amd64-BASIC-CLOUDINIT-zfs.raw -C -k ~/.ssh/vms_id_ed.pub xBuild141
qemu-img resize -f raw xBuild141/disk0.img 35G # resize before first boot
vm start xBuild141
</pre>
==== Bastille vnet + vm-bhyve vnet ====
<pre>
finch:~/vms/xigmanas# ifconfig bridge
ifconfig: interface bridge does not exist
finch:~/vms/xigmanas# ifconfig bridge create
bridge0
finch:~/vms/xigmanas# ifconfig bridge0 name em0bridge
em0bridge
finch:~/vms/xigmanas# ifconfig em0bridge addm em0 up
finch:~/vms/xigmanas# ifconfig em0bridge
</pre>
Share bastille's bridge with bhyve. (manually setup the switch)
<pre>
vm switch create -t manual -b em0bridge public
</pre>

==== vm-bhyve restart ====
since vm-bhyve is within finch chroot, the mount points need to be re-mounted within the chroot
<pre>
zfs list -t filesystem | awk '/vms/ { print $1}' | xargs -n1 zfs mount

</pre>