ssincan / kvm-ip-zynq

KVM over IP Gateway targeting Zynq-7000 SoC

KVM over IP Gateway

A KVM over IP Gateway, enabling transparent remote access over an IP network, implemented on the Xilinx Zynq-7020 SoC (Zybo Z7-20 development board).

2 Minute Demo Video

Architecture

The Zynq Programmable Logic implements DVI capture and JPEG encoding. The JPEG-encoded image is transferred to the DRAM via the AXI HP Ports, and served by busybox httpd running under PetaLinux in the Zynq Processor Subsystem.

In the opposite direction, mouse events are captured in the browser using the Pointer Lock API, sent as requests to the HTTP server, and piped to a HID Gadget implementing a mouse.

Keyboard functionality is not yet implemented. (I guess this makes it a KVM over IP Gateway.)

State of Code Base

The SW side of the project (CGI scripts and browser-based client) should be regarded as a PoC or "technology demonstrator". It's mostly pre-existing code, glued together in the simplest (if a bit hackish) way possible.

Tools

Built and tested with Xilinx Vivado 2018.3 (WebPACK) and PetaLinux 2018.3.

Building the Vivado Project

Open Vivado, change the current working directory to gateware, and run source ./zybo_z7_kvm_prj.tcl from the Vivado TCL Console. This will create the Vivado project under gateware/zybo_z7_kvm. You can then run all implementation and/or simulation steps.

Simulation

The behavioral simulation is already configured in the project generated above. After a some time (at least 150 milliseconds simulation time, see gateware/video_capture/sim/video_capture_tb.vhd), JPEG images will be generated in gateware/zybo_z7_kvm/zybo_z7_kvm.sim/sim_1/behav/xsim.

The input images are included in gateware/video_capture/sim/stim_img.zip and extracted by gateware/zybo_z7_kvm_prj.tcl. They have been generated with the Python script found in gateware/video_capture/sim/gen_stim_img.py.

Building the PetaLinux Image

The PetaLinux project is under petalinux/zybo_z7_kvm_plnx. Follow the instructions in Xilinx UG1144 in order to build the project.

Upon extracting ROOTFS to the corresponding SD Card partition, you will need to add the files included in petalinux/rootfs_overlay to the same partition. TODO: include them in a Yocto layer and have petalinux-build do this automatically.

The network interface is configured with a static IPv4 address, 192.168.2.10. This can be changed from the PetaLinux project configuration.

The root password for the image is the PetaLinux default one.

Initialization

Boot the development board from the SD Card. SSH to the PetaLinux instance and source /home/root/initmouse.sh. TODO: source it automatically at PetaLinux boot.

Future Development

Area reduction

Ideally the project would fit in Zynq-7010 to enable porting to even cheaper hardware.

Remove Clocking Limitations

DRP can be used to reconfigure the MMCM in the DVI2RGB IP in order to dynamically support different clock ranges.

Implement Keyboard

The solution which would work with the cuurrent HW is a composite Mouse + Keyboard HID Gadget.

Implement Virtual Storage

Some more expensive commercial KVM over IP Gateways can emulate a removable drive, populated with a disk image that is controlled by the client. This can be done with the Zybo-Z7 USB OTG port (see equivalent ZC702 example here), but that means either creating a composite device with Mouse + Keyboard + Mass Storage, or finding another solution for Mouse + Keyboard, which brings us to the next point.

Alternate Mouse/Keyboard Implementation

A potential solution to free up the USB OTG port for use as a Mass Storage Peripheral, is to use one or two 3.3V Pro Micro boards to implement the HID devices, as shown e.g. here. These would receive the mouse/keyboard events from the Zybo-Z7 board through an UART, connected via a PMOD port.

License

The gateware (FPGA bitstream) part of this project is licensed under the MIT License - see the LICENSE.md file for details.

There are some less permissive exceptions to this for reused components in the SW side. These are noted in the corresponding source files (e.g. petalinux/zybo_z7_kvm_plnx/project-spec/meta-user/recipes-apps/hidgadgettest/files/hidgadgettest.c is a derivative of hid_gadget_test).

Acknowledgments

• Michal Krepa for the mkjpeg IP.
• Digilent Inc. for the DVI2RGB IP.
• Tom Hebb for memdump, used in the CGI scripts to serve the images from RAM.

kvm-ip-zynq's Issues

use in single channel mode (instead 4)

Hi
I am trying to convert the project to single channel mode. I have already successfully converted and tested the project in dual channel mode with 1280*720 resolution and 1000/7 MHz clock and I'm aware of details on how to do that. Here are some changes I have made to the project to work in single channel mode:

-changed DRAM and PS HP ports count to 1.
-changed num_chan to 1.
-changes needed in striped_encoders.vhd for axi4lite_reg_file and and address set process.
-changed C_MAX_LINE_WIDTH to 2048 in JPEG_PKG.vhd (more than needed)
-changed res_x_nopad_ch calculate method in image_stripe.vhd, since it was correct for 2 channels or more. now it is set to res_x_nopad_ch(i) <= shift_right(unsigned(res_x_in), 0); or res_x_nopad_ch(i) <res_x_in; which is correct for 1 channel.
-in buffered_encoder I also changed RES_X_MAX to 2048 and JPEG_FIFO_DEPTH to 65536/ AXI_WORD_LEN; , although i think they are not needed, since i had correct output with default valeus (512 and 16384/ AXI_WORD_LEN) with 2 channels and 1280*720 input and buffers where enough. any way i changed them to larger values.

Now the problem is that for single channel mode simulation (output and picture size) is correct, but on hardware, when I peek image size from buffer, it is always equal to 2^32-1 (all bits one). I think image is captured from hdmi correctly (since RGB LED shows correct state), but jpg encoder has problem.

Do you have any idea why is this happening on hardware? may it be related to encoder clock frequency (1000/7) is not adequately high? simulation is using same clocks (74.25 for pixel clock and 1000/7 for encoder clock) and the result is correct! I have checked the code and have no idea what is going wrong.

Thanks for the great project :)