We have a "control port" which appears via the USB PROG port as a "usb serial port".

The Atlys board also has a "USB-UART" connect. It would be nice if the control port was also connected to this. Allowing control / debugging of the system when the Cypress firmware programming fails.

http://www.digilentinc.com/Data/Products/ATLYS/Atlys_ISE_GPIO_UART.zip

Makefile, cmake or other system which builds all of the following;

• CDC Firmware
• FGPA Firmware
• Tools needed to load firmware such as;
  • fxload
  • fgpa download tool???

The Xilinx Zynq-7000 series devices are a combined ARM processor with a "series 7" FPGA. The Digilent ZYBO is a very cheap development board $189 USD for these devices and even has a dual Sink/Source HDMI port.

We should port the HDMI2USB firmware to this device. It would also be a good trial run for porting to the Novena device as the AXI interconnect between the ARM processor and the FPGA is used in both devices.

The ZYBO (Zynq Board) is a feature-rich, ready-to-use, entry-level embedded software and digital circuit development platform built around the smallest member of the Xilinx Zynq-7000 family, the Z-7010. The Z-7010 is based on the Xilinx All Programmable System-on-Chip (AP SoC) architecture, which tightly integrates a dual-core ARM Cortex-A9 processor with Xilinx 7-series Field Programmable Gate Array (FPGA) logic. When coupled with the rich set of multimedia and connectivity peripherals available on the ZYBO, the Zynq Z-7010 can host a whole system design. The on-board memories, video and audio I/O, dual-role USB, Ethernet and SD slot will have your design up-and-ready with no additional hardware needed. Additionally, six Pmod connectors are available to put any design on an easy growth path.

The ZYBO provides an ultra-low cost alternative to the ZedBoard for designers that don't require the high-density I/O of the FMC connector, but still wish to leverage the massive processing power and extensibility of the Zynq AP SoC architecture.

The ZYBO is compatible with Xilinx’s new high-performance Vivado Design Suite as well as the ISE/EDK toolset. These toolsets meld FPGA logic design with embedded ARM software development into an easy to use, intuitive design flow. They can be used for designing systems of any complexity, from a complete operating system running multiple server applications in tandem, down to a simple bare-metal program that controls some LEDs. For systems that require an operating system, Digilent provides an out-of-the-box Linux solution specifically targeted to run on the ZYBO, complete with documentation describing how best to mold it to suite your purposes. These Xilinx tools and Linux solution are all available to use at no additional cost with the ZYBO.

Sometimes presenters display has poor color choices, like white text on grey background.

Inverting the display can dramatically improve the readability. I am guessing there are some other simple changes that could be implemented. like dropping color depth or gray scale?

The build currently contains a large number of warnings:

  Number of warnings: 341

These should be fixed, or suppressed.

Running a HDMI 640x480 input (Xbox 360) results in a corrupted display. As a result it is difficult to fix the resolution so that it appears properly.

Connecting Xbox to HDMI TV results in different mode selection to when it is connected to Atlys board. When connected to Atlys board, Xbox detects display as DVI (a PC monitor) instead of a TV.

Eventually after navigating the menus to fix the resolution, the Xbox sets the resolution correctly. However this takes a couple of attempts to make it work.

HDMI passthrough to a TV at 640x480 only works intermittently, and at 1024x768 passthrough does not work, however the capture works properly.

What I believe should happen for sub-1024x768 resolutions, is that the input video displays in the top-left corner, at the reduced resolution, instead of trying to display all the pixel data it possibly can.

The Novena laptop has a LX45 Spartan-6 chip inside it. The device is also completely open hardware and it would be an excellent platform to support.

This will require designing of a hardware daughterboard to connect the pins on the Spartan-6 to a HDMI connector (probably through a HDMI buffer / ESD protection device).

There is probably a lot of code / design / etc that could be shared between this and the Zybo port.

On the production Numato board the Cypress doesn't have a dedicated EEPROM for the Cypress.

Instead the FPGA is will emulate an EEPROM and pull the Cypress from the same flash that the FPGA boots from (as there is plenty of space in that).

Two HDMI inputs, two HDMI outputs.

Control which input goes to which output via both:

• D-Pad control
• CDC Commport

Latest firmware from here gives a black image:

https://www.dropbox.com/sh/s5zw70u754lpbmv/N95myLaoJT

91da91a4eada6a72c1dedf02d3d732ee771b5535  hdmi2usb.bit
5a5cf80748633448eaecf3d0b88ec50015034153  hdmi2usb.hex

"Archive" MJPEG firmware gives correct image:

e540d3210f7c2fc71d1427c0e12528a86aeae525  hdmi2usb_mjpeg.bit
ce380c747052530198c134527392c1c466dd6448  hdmi2usb_mjpeg.hex

LEDs LD7 - LD0 show: on on on dim off off off on

Pressing UP changes LEDs to show: off off on dim off off off on

make doc should generate doxygen style documentation of the VHDL code.

The HDMI2USB presents multiple USB devices to the host computer. By surfacing a serial number in the device information, the host computer can associate them together.

The same serial number should be presented on both the UVC device and the CDC device. Device DNA on the Spartan 6 chip gives us a unique serial number we can use for this. We just need to get the value up into the Cypress endpoint information.

This will get more important as the HDMI2USB grows new features like postage stamp previews of the video inputs, serial ports, etc. Having the serial number we can use udev to end up with something like the following;

/dev/hdmi2usb/<serialnumber>/video
/dev/hdmi2usb/<serialnumber>/control
/dev/hdmi2usb/<serialnumber>/video-preview-input1
/dev/hdmi2usb/<serialnumber>/video-preview-input2
/dev/hdmi2usb/<serialnumber>/rs232-raw
/dev/hdmi2usb/<serialnumber>/rs422-raw
/dev/hdmi2usb/<serialnumber>/rs422-device-1
/dev/hdmi2usb/<serialnumber>/rs422-device-2
....

USB Serial Number in udev under Linux

It would be really great if we could get that serial number to appear in a field that udev has access (for both the serial and video device). Then on linux we can provide a udev rule which creates a directory which contains both the video and serial device. Much easier than trying to figure out which serial/video devices are associated with a HDMI2USB device. This would also make multiple HDMI2USB devices easier to work with.

Information I have found on doing this;
https://wiki.debian.org/udev
http://hintshop.ludvig.co.nz/show/persistent-names-usb-serial-devices/

udevadm info -a -n /dev/ttyUSB1 | grep '{serial}'

Device DNS on Spartan 6

http://www.xilinx.com/support/documentation/user_guides/ug380.pdf

JTAG Access to Device Identifier

The FPGA's internal device identifier, plus any values shifted in on the
DIN input, can be
read via the JTAG port using the private ISC_DNA command. This requires
the
ISC_ENABLE to be loaded before the ISC_DNA command is issued.
Bit 56 of the identifier, shown in Figure 5-14, appears on the TDO JTAG
output following
the ISC_DNA command when the device enters the Shift-DR state. The
remaining Device
DNA bits and any data on the input to the register are shifted out
sequentially while the
JTAG controller is left in the Shift-DR state. When this operation is
complete, the
ISC_DISABLE command should be issued.

iMPACT Access to Device Identifier

The iMPACT software in ISE 10.1 (and later) tools can also read the device DNA value.
readDna -p

From http://www.xilinx.com/support/documentation/sw_manuals/xilinx12_1/spartan6_hdl.pdf page no 94.

Device DNA

This element allows access to a dedicated shift register that can be loaded
with the Device DNA data bits (unique ID) for a given Virtex®-6 device. In
addition to shifting out the DNA data bits, this component allows for the
inclusion of supplemental bits of your data, or allows for the DNA data to
rollover (repeat DNA data after initial data has been shifted out).

To access the Device DNA data, you must first load the shift register by
setting the active high READ signal for one clock cycle. After the shift
register is loaded, the data can be synchronously shifted out by enabling the
active high SHIFT input and capturing the data out the DOUT output port.
Additional data can be appended to the end of the 57-bit shift register by
connecting the appropriate logic to the DIN port. If DNA data rollover is
desired, connect the DOUT port directly to the DIN port to allow for the same
data to be shifted out after completing the 57-bit shift operation. If no
additional data is necessary, the DIN port can be tied to a logic zero. The
attribute SIM_DNA_VALUE can be optionally set to allow for simulation of a
possible DNA data sequence. By default, the Device DNA data bits are all
zeros in the simulation model.

The HDMI2USB inputs can be switched to a test image. It would be nice if this image was prettier and/or had some type animation in it.

A number of Mac's didn't work with the board.

Should support USB 2.0 backwards compatible.

Support transfer of raw frames rather than MJPEG stream.

Take a look at https://github.com/mossmann/daisho/tree/master/sw/fpga/common/usb3

Daisho USB Core 

This is an open source USB 3.0 core.  It can be used to implement a USB device
on an FPGA with a USB 3.0 transceiver such as the TUSB1310A.  It can also
operate in USB 2.0 mode with a ULPI transceiver. 

This core is being developed for the Altera Cyclone IV.  Operation on other
FPGA platforms may be possible with some work. 

Planned functionality includes device (not host) operation in USB 2.0 and 3.0
(including SuperSpeed) modes. 

The core is part of Project Daisho but may be used on other platforms.

The Numato board uses PIC18F series chip. This chip is supported by sdcc compiler.

PIC pin configuration is found at https://docs.google.com/a/mithis.com/spreadsheet/ccc?key=0ApVlNZ_Rvdw6dGM1dU5wUWVORm9qY2hLUW5waF9NRVE&usp=drive_web
Power Supply configuration found at https://docs.google.com/a/mithis.com/spreadsheets/d/1V04Nr9teP0HtJ4cOB4p0Ok7I8uUZKsOyHPrfGWu1LF8/edit#gid=0

The PIC / FPGA / Cypress reset and JTAG configuration can be at https://docs.google.com/a/mithis.com/document/d/13sMIqRzbgp4ZS9HvfSiXpRa4pKYtOEjhzU0h4RfXmgs/edit

Optimisation of the MJPEG core in the HDMI2USB firmware to support full frame rate recording.

Current performance is roughly 720p@15fps minimum requirement would be 720p@30fps. We'd really like it to reach somewhere between 1080p@30Hz --> 1080p@120Hz.

The current HDMI2USB core was generated from the mkjpeg project at OpenCores and found at hdl/jpeg_encoder code.

NeroJTAG seems to be the dominant open source JTAG communication protocol. It would be nice if the PIC18F series devices (which have inbuilt USB controllers) had support for this.

http://www.makestuff.eu/wordpress/software/nerojtag/

NeroJTAG is used by fpgalink.

Exposing the EDID information and other I2C busses via the USB could be useful for debug an experimentation.

One solution would be to emulate the i2c_tiny_usb device. A driver for this device already exist in the Linux kernel. See http://www.harbaum.org/till/i2c_tiny_usb/index.shtml for more information.

The http://digilentinc.com/Products/Detail.cfm?NavPath=2,648,870&Prod=VMOD-MIB adds another 4 HDMI ports to the Digilent board, but they are the HDMI-Micro-D connectors which are hard to get cables for.

Would be awesome to create a version which has HDMI or DVI connectors instead.

VGA is going to be around for a long time, we really need to continue to have a VGA input system.

Use instead of USB

Please check that HDMI2USB works with the following set ups;

Set Up 1

• Capture PC: Windows 7
• Input PC: Windows 7

I only included this set up as I believe it is what you are testing with at the moment right?

Set Up 2

• Capture PC: Ubuntu Precise
• Input: Ubuntu Precise computer

Set Up 3

• Capture PC: Ubuntu Precise
• Input: Windows 7 computer

Set Up 4

• Capture PC: Ubuntu Precise
• Input: Mac OS X computer

Set Up 5

• Capture PC: Ubuntu Precise
• Input: Raspberry PI

I can send you a Raspberry Pi if you don't have one.

Set Up 6

• Capture PC: Ubuntu Precise
• Input: PS3 or Xbox 360 or Nintendo Wii

This is a low priority, but I would like to see what happens - I think HDCP is probably going to make it busted...

http://www.usb.org/developers/docs/devclass_docs/audio10.pdf

Write a script which captures and uploads to the website.

If the display is static, it is hard to tell if everything is functioning.

it would be handy (maybe?) to be able to enable something dynamic that can be checked for. maybe even 1 pixel, or small block in the corner that pulses.

Currently we are using a VID:PID that was chosen randomly, this is probably bad.

From http://www.oshwa.org/2013/11/19/new-faq-on-usb-vendor-id-and-product-id/

USB vendor IDs (VID) and product IDs (PID) are 16-bit numbers used to identify USB devices to a computer or other host. Each vendor ID is assigned by the USB Implementers Forum to a specific company, which in turn assign a PID to individual products. The VID and PID are then embedded in the product and communicated to the computer when the device is plugged in, along with text strings describing the vendor and product and additional descriptors about the communication protocols supported by the device.

Options appear to be;

If you’re using an off-the-shelf chip for USB communication, you may be able to use the VID/PID of that chip (e.g. with FTDI chips). OpenMoko has opened up their VID for use by free and open-source software and hardware communities. Here’s one attempt to get a VID for open-source projects. Ian Lesnet also gave a good talk on the USB VID/PID situation (slides) at the 2012 Open Hardware Summit.

Should do the following;

• Read everything it can out of the CDC
• Capture a screenshot of each HDMI input
• Get MD5 sums of the firmware loaded onto the two devices? - Is that even possible?
• Capture dmesg, loaded modules, etc
• Capture system version

Basically it should collect a really nice debug report which makes it easy to figure out what the problem is.

After l've followed the instructions on:
https://github.com/timvideos/HDMI2USB/wiki/Digilent-Atlys-Board:-Loading-Firmware
Up to Stage 3, step 3, when I plug the USB cable back into the PROG port, on my Linux box I get:

[8552952.824097] usb 1-1.6.2: new full-speed USB device number 120 using ehci-pci
[8552953.243713] usb 1-1.6.2: device not accepting address 120, error -32
[8552953.243830] hub 1-1.6:1.0: unable to enumerate USB device on port 2

The HDMI2USB test screen is output on the HDMI OUT, and if I plug in a PC to an HDMI IN port, I can select the input which is correctly displayed on HDMI OUT.

I've tried plugging the USB into a screen connected to my PC, and into a laptop. USB devices on my PC:

Bus 004 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 004 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 002: ID 045e:0750 Microsoft Corp. Wired Keyboard 600
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 059: ID 04b4:6560 Cypress Semiconductor Corp. CY7C65640 USB-2.0 "TetraHub"
Bus 001 Device 072: ID 045e:0024 Microsoft Corp. Trackball Explorer
Bus 001 Device 071: ID 04b4:6560 Cypress Semiconductor Corp. CY7C65640 USB-2.0 "TetraHub"
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Firmware is commit 2ddc2c7cf4326bd860e0e4f30206b2d49093873b in the HDMI2USB-firmware-prebuilt git repo.

Should test the following;

• edid parsing
• jpeg encoder
• USB stack

make test (or equivalent) should run the tests and confirm the success.

It would be nice if firmware for the PIC18F series devices which pretended to be a Altera USB Blaster existed.

• Altera USB Blaster is a popular USB to JTAG interface that is supported via a wide variety of tools.
• A lot of the PIC18F series have an inbuilt USB2.0 port.
• Open Source firmware for the Cypress EZ-USB FX2 (which is an 8051 process) already exists. This would just need to be ported to the PIC18F series.
  • The project was originally called usbjtag but has been unmaintained for a long time.
  • The most advanced fork appears to be by the fpga4u guys and can be found at http://fpga4u.epfl.ch/wiki/FX2

Do some research to see if this actually exists before writing the code!

Find out what chips are used in them and what type of cost they are.

The HDMI2USB should appear as a display to the computer, even when there is no display plugged into it.

Something like memcheck or similar.

Performance things to check;

• Write a large contiguous block.
• Read a large contiguous block.
• Write at random locations.
• Read at random locations.
• How interleaving read/write operations affects performance.
• Compare "width" (1 byte, 2 bytes, 4 bytes, 8 bytes).

"Integrity" checking.

• http://opencores.org/websvn,filedetails?repname=ion&path=%2Fion%2Ftrunk%2Fsrc%2Fmemtest%2Freadme.txt&rev=193
• http://www.memtest86.com/technical.htm
• https://github.com/Reisswolf/ztex/blob/master/examples/usb-fpga-1.15/1.15d/memtest/fpga/memtest.vhd
• http://www.barrgroup.com/Embedded-Systems/How-To/Memory-Test-Suite-C

Integrity checking is related to performance. Bump the clock speed up - do integrity checking, repeat until you have errors.

JoelW has some good information about the MIG on the Atlys board at http://www.joelw.id.au/FPGA/XilinxMIGTutorial

Currently the EDID information is parsed through from the output monitor but stripped of all resolutions apart from 1024x768.

It is probably better we just have our own EDID information.

Wire shark is the tool for doing debugging of protocols, would be great if the system had a raw mode which dumped the HDMI data in format suitable for wire shark.

After loading the hdmi2usb.hex from Drop Box/freelancehandover/cypress/uvc+cdc/output/hdmi2usb.hex

The CDC port doesn't work under Linux, it has the following error in dmesg

[2324637.624638] cdc_acm 2-1.2.2:1.2: This device cannot do calls on its own. It is not a modem.
[2324637.624648] cdc_acm: probe of 2-1.2.2:1.2 failed with error -22

We are quickly getting multiple types of "boards" that the HDMI2USB firmware should support.

The following spreadsheet contains the information about how they are wired together and which modules are currently supported. It would be nice to auto-generate the details from it.
https://docs.google.com/a/mithis.com/spreadsheets/d/10vNcsOAxnuiwc5diespjIepMySxhR0iVZfYxouq4p-E/edit

Need to:

• Remove all temporary files
• Remove all files produced by the compiler apart from the .bit files
• Rewrite history to include the above
• Add .gitignore to include the files to ignore
• Make R1 and subsequent releases into proper tag/branchs.

We need to support;

• Multiple host interfaces;
  • Our current system uses a Cypress FX2 over USB, it is likely we want to support different (cheap) USB solutions.
  • We are have @tariq786 working on an Ethernet host interface.
  • To support the Zybo and Novena devices we'll need a AXI interface too.
• Variable number of capture interfaces;
  • At a basic level, the Atlys and Numato boards both have 2 input and 2 outputs.
  • However, both support the VGA expansion board which adds a third VGA input.
  • There is also the VmodMIB which in theory adds another 4 HDMI bi-directional ports.
  • On the Numato conference board, there is a possibility of the GTP transceivers being used which could support DisplayPort and SDI but use a very different VHDL/Verilog model.
  • The Zybo device only has a single bi-directional HDMI port, but the AXI interface adds a lot of possibility of "virtual" interfaces.
  • The EDID / CEC stuff is probably usefully common between all these systems.
• Variable number (and type) of expansion ports
  • The Atlys and Numato Conference board have compatible VHDCI expansion ports.
  • The Atlys and Zybo have compatible PMOD ports.
  • The Numato Conference board has the GTP transceivers on a DVI port.
  • The Novena uses SATA connectors.
• Various debug / control routing and interfaces
  • It would be nice to get it up the CDC port over USB.
  • It would be nice to get it out on a serial ports, pins, etc.
  • It would be nice to get it out the USB-UART on the Atlys board.
• There are a bunch of subsystems which should be common to all boards though.
  • There should be a debug system which everything uses and automatically exposes the information to everyone.
  • Scalers, JPEG encoders are probably just generally useful.

Since we now capturing the whole frames into RAM, it should be possible to ship up raw frames at a low frame rate to the system.

I believe the UVC spec supports raw YUV and RGB frames.

When a vertical line is on a block boundary, JPEG can sometimes discard the whole line - this is mostly apparent on fine text capture.

Investigate ways to mitigate this problem and add tests to prevent this problem from reoccurring.

Raspberry pi has support, would be good if we could duplicate it.

Using firmware from Drop Box/freelancerhandover my laptop detects the following valid resolutions.

HDMI1 connected 1024x768+1366+0 (normal left inverted right x axis y axis) 0mm x 0mm
   1024x768       60.0*+
   800x600        60.3     56.2  
   848x480        60.0  
   640x480        59.9  

A number of users have reported seeing the following using fxload. The same firmware images are successful when using libFPGALink. @ajitmathew and @shenki have experienced the same symptoms.

[ 4082.081760] usb 3-1: new high-speed USB device number 5 using xhci_hcd
[ 4082.099278] usb 3-1: config 1 interface 0 altsetting 0 bulk endpoint 0x1 has invalid maxpacket 64
[ 4082.099289] usb 3-1: config 1 interface 0 altsetting 0 bulk endpoint 0x81 has invalid maxpacket 64
[ 4082.101581] usb 3-1: string descriptor 0 read error: -32
[ 4082.101595] usb 3-1: New USB device found, idVendor=1443, idProduct=0007
[ 4082.101601] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 4333.753510] usb 3-1: USB disconnect, device number 5
[ 5288.065190] usb 3-1: new high-speed USB device number 6 using xhci_hcd
[ 5288.082848] usb 3-1: config 1 interface 0 altsetting 0 bulk endpoint 0x1 has invalid maxpacket 64
[ 5288.082859] usb 3-1: config 1 interface 0 altsetting 0 bulk endpoint 0x81 has invalid maxpacket 64
[ 5288.085585] usb 3-1: string descriptor 0 read error: -32
[ 5288.085601] usb 3-1: New USB device found, idVendor=1443, idProduct=0007
[ 5288.085607] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 6689.993968] usb 3-1: USB disconnect, device number 6
[ 6691.756616] usb 3-1: new full-speed USB device number 7 using xhci_hcd
[ 6691.759252] usb 3-1: Device not responding to set address.
[ 6691.963036] usb 3-1: Device not responding to set address.
[ 6692.164121] usb 3-1: device not accepting address 7, error -71
[ 6692.275953] usb 3-1: new full-speed USB device number 8 using xhci_hcd
[ 6692.278503] usb 3-1: Device not responding to set address.
[ 6692.482280] usb 3-1: Device not responding to set address.
[ 6692.683371] usb 3-1: device not accepting address 8, error -71
[ 6692.795242] usb 3-1: new full-speed USB device number 9 using xhci_hcd
[ 6692.797829] usb 3-1: Device not responding to set address.
[ 6693.001626] usb 3-1: Device not responding to set address.
[ 6693.202716] usb 3-1: device not accepting address 9, error -71

Currently we use the closed source Kiel UV2 software for compiling the Cypress firmware.

We'd like to use open source tools instead, the sdcc compiler has support for the 8051 chip inside the chip and a number of people have already done a lot of the work. Take a look at the links below;

Links

• https://www.google.com.au/search?q=sdcc+cypress+ez+usb+firmware
• http://www.linux-usb.org/ezusb/
• http://braiden.org/?p=59
• https://github.com/hansiglaser/ezusb-firmware
• http://allmybrain.com/2009/04/14/fx2-cystream-throughput-test-with-sdcc-and-fx2lib/