View source for Proton Calorimetry/Equipment/FPGAs DDC232

=Useful Material=
==Technical Documents==
*[https://www.ti.com/lit/gpn/ddc232 DDC232 datasheet]. This contains essential information from which the FPGA design was created.
*[https://reference.digilentinc.com/reference/programmable-logic/nexys-video/reference-manual Nexys Video manual].
*[https://www.xilinx.com/support/documentation/ip_documentation/clk_wiz/v5_3/pg065-clk-wiz.pdf Xilinx Clocking Wizard manual].
*[https://www.xilinx.com/support/documentation/ip_documentation/ila/v6_2/pg172-ila.pdf Xilinx Integrated Logic Analyser manual].
*[https://www.xilinx.com/support/documentation/ip_documentation/fifo_generator/v13_1/pg057-fifo-generator.pdf Xilinx FIFO Generator manual].
*[https://www.ftdichip.com/Support/Documents/DataSheets/ICs/DS_FT2232H.pdf FTDI chip datasheet].
*[https://www.intra2net.com/en/developer/libftdi/index.php libFTDI]. C++ library used to communicate with the FTDI chip.
*[https://steckdenis.be/post-2016-06-25-fast-usb-connection-on-the-nexys-video-using-ft2232h.html FTDI tutorial]. Useful example on how to use the Nexys Video FTDI chip with libFTDI.
*[http://embedded-funk.net/running-libftdi-under-windows/ libFTDI on Windows].
*[https://www.nandland.com/vhdl/modules/module-uart-serial-port-rs232.html NandLand UART modules]. This website also contains several useful tutorials for learning FPGA design.
*[https://reference.digilentinc.com/reference/programmable-logic/zybo-z7/reference-manual Zybo Z7 manual].

==Presentations==
*I/O signals and operation of TI DDC232CK based on datasheet information. Slides available [http://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2020/SS090620_DDC232.pptx here].
*Results of simulation of FPGA design discussed [http://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2020/SS080720_DDC232_SIM.pptx here].
*UART interface between FPGA and PC discussed in [http://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2020/SS150720_UART_SIM.pptx this presentation].
*First operational tests discussed in [http://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2020/SS220720_DDC232_TEST.pptx this presentation].
*DDC232 design features and demo given in [http://www.hep.ucl.ac.uk/pbt/wikiData/presentations/2020/SS290720_DDC232_DEMO.pptx this presentation].

=Background=
==DDC232==
A Texas Instruments (Dallas, Texas, United States) DDC232CK (DDC) was chosen as the current-input analogue-to-digital converter for its speed, large dynamic range and low power requirements. It is capable of measuring the currents of up to 32 photodiodes with an adjustable integration time (160μs–1s) and full-scale range (FSR, 12pC–350pC). Each of the 32 inputs on the DDC has two integrators, allowing for zero-deadtime measurements: while one integrator digitises and transfers data, the other measures the input current. The DDC is housed on a compact custom circuit board manufactured by CosyLab (Ljubljana, Slovenia), where the charge collected by a photodiode is split across two DDC inputs to give 16 photodiodes per DDC. The role of each digital signal is described below:

*CLK (input): 10 MHz clocking signal that is used to time the internal operations of the DDC, including generation of DVALID.
*CONV (input): signal that controls integration, the time period of which is equal to the integration time. When this signal toggles, the integrator of each input switches.
*DIN_CFG (input): serial data stream of the 12-bit sequence used to set key parameters of the DDC, namely the FSR and measurement precision (16-bit or 20-bit).
*CLK_CFG (input): 20 MHz clocking signal used to time sending and reading of DIN_CFG.
*RESET (input): Asynchronous active-low reset signal for the DDC to revert it to its power-up state.
*DCLK (input): 20 MHz clocking signal used to time sending and reading of DOUT.
*DVALID (output): active-low signal used to indicate that data is ready to be read on DOUT.
*DOUT (output): serial data stream of the 640-bit sequence (when in 20-bit precision mode) containing measurements of the 32 inputs.
*DIN (input): serial data input to the DDC used to daisy-chain other DDCs.



==Configuration==
The DDC must first be configured with a 12-bit sequence of data sent on DIN_CFG, where:
*Bits 11-9 correspond to the 3 FSR bits, allowing for 8 different dynamic ranges; 000 = 12.5 pC, 001 = 50 pC, 010 = 100 pC, 011 = 150 pC, 100 = 200 pC, 101 = 250 pC, 110 = 300 pC, 111 = 350 pC. These correspond to the maximum charge that can be integrated in the photodiodes. 350 pC is typically chosen to minimise risk of saturation of inputs.
*Bit 8 corresponds to the resolution of output data: 0 = 16-bits, 1 = 20-bits. 20-bit resolution is chosen for better precision, at the price of slower readout (due to more bits of data requiring shifting out).
*Bit 7 corresponds to the device version. For the DDC232CK, this bit is set to 1.
*Bit 6 corresponds to a DDC internal divider of the CLK signal. This is set to 0 for no division.
*Bits 5-1 are empty bits set to 0.
*Bit 0 corresponds to the diagnostic test mode setting: 1 = on, 0 = off.
This gives a configuration input for normal operation of: 111110000000. Note that all vectors of data in this design are “little-endian”, in which the bits are labelled from the most-significant bit (MSB) to the least-significant bit (LSB) and are sent to and from devices MSB first. For the configuration input, this means bit 11 is sent first and bit 0 is sent last. After the configuration data is sent, a 640-bit (when in 20-bit mode, which is equal to the number of bits sent during a measurement readout cycle) read-back is sent to confirm configuration settings and test data output. The 640-bits are a 320-bit sequence sent twice, which contains the 12 configuration bits that the DDC received, a 4-bit revision ID (0001), 244 zeros and then a 70-bit test pattern, used as an extra check. In hexadecimal, the test pattern is: 30F066012480F69055.

After the DDC is powered up and power supplies have stabilised, a reset pulse of width t_RST = 1 μs must be sent and after t_WTRST = 2 μs, the configuration data is sent through DIN_CFG on the rising edges of CLK_CFG, to be read by the DDC on the falling edges of CLK_CFG. t_STCF and t_HDCF (both 10 ns) are the minimum times required for DIN_CFG to be valid before and after falling edges of CLK CFG respectively. t_WTWR = 2 μs later, configuration read-back begins on DOUT where the pattern is read on the rising edges of DCLK, after which CONV is strobed (i.e. high for one CLK cycle) to begin integration.

==Integration==
After CONV is strobed at the end of the configuration read-back, it begins alternating with a time period t_INT, which is the integration time set by the user. Toggles in CONV represent transitions in the dual-integrators, where one side completes integration and begins measurement, reset and auto-zeroing while the other side, currently idling after having finished these three tasks, begins integration. Measurement, reset and auto-zeroing take t_MRAZ = 1612 ± 2 CLK cycles and DVALID is asserted after t_CMDR = 1382 ± 2 CLK cycles, signalling that data ready to be shifted out on DOUT.

==Data Transfer==
Once DVALID is asserted, photodiode data can be shifted out to the FPGA on the rising edges of DCLK. Each of the 32 DDC inputs is represented by a 20-bit number, which can then be converted to a charge in pC. The simplest method of data retrieval is after DVALID goes low and before the next CONV toggle. A key requirement for the DDC to be operated in continuous mode with this method, i.e. with zero dead-time between measurements (not to be confused with the FPGA continuous/triggered acquisition settings), is that the integration time must be sufficient to complete measurement, reset, auto- zero and shifting out of data before the next CONV toggle. Otherwise, the DDC enters a non-continuous mode where integration stops until the aforementioned processes have finished on both input integrators.

Regardless of whether the FPGA is in continuous or triggered acquisition modes, each measurement sent by the DDC is read, though they are not always sent to the PC (see below). The FPGA has a 640-bit register that is overwritten every time new data arrives. Before being sent to the PC, a measurement is first sent to a FIFO interface on the FPGA. The FIFO was added to the design using the Xilinx FIFO Generator and has the following signals:

*WR_EN: FIFO write enable. When high, a byte is written to the FIFO on clock rising edges.
*RD_EN: FIFO read enable. When high, the oldest byte in the FIFO is read on clock rising edges. 
*FULL: High when the FIFO is full.
*EMPTY: High when the FIFO is empty.
*WR ACK: High when a byte has been written successfully.
*VALID: High when a byte has been read successfully.
*OVERFLOW : High if FIFO is written to when full and causes the Zybo RGB LED to turn blue. 
*UNDERFLOW: High if FIFO is read when empty and causes the Zybo RGB LED to turn blue.

A 640-bit measurement received by the FPGA is written to the FIFO in byte-sized pieces, such that the FPGA is quickly ready again to read new data. The measurement is then sent from the FIFO to the PC over the universal asynchronous receiver-transmitter (UART) protocol, which sends data a byte at a time (hence the storage of a measurement in bytes in the FIFO). The principle of the UART protocol is shown in Fig. 16 and the key user settings are:

*Baud rate: the number of bits transferred per second (including start and stop bits). This is set to 921600, which is often the maximum standard baud rate for serial ports.
*Data bits: the number of bits of data sent in a packet. This is set to the maximum number, 8.
*Stop bit: the number of bits at the end of the data bits to denote the end of data. This is set to the minimum, 1.

Key signals for the UART transmitter used to send data from the FPGA to the PC:
*TX_DV: Data valid signal sent to the UART transmitter to state that data is ready for transmission. Functionally, this serves the same purpose as the FIFO VALID signal.
*TX_ACTIVE: High when the UART transmitter is sending data to the PC.
*TX_DONE: High for one clock cycle after a byte is sent.
*TX_SERIAL: Actual data stream sent to the PC.

In triggered mode, the next measurement read by the FPGA is stored into the FIFO and then sent over UART. Subsequent measurements are only read, to be overwritten by the next until the button is pressed again. At the falling edge of DVALID, DCLK begins to shift in data on DOUT, after which the measurement is stored in bytes into the FIFO. The UART transmitter then sends data one byte at time. When the FPGA is in continuous mode, the next read measurement is sent to the FIFO as soon as the UART transmitter has finished sending the last measurement (i.e. when the FIFO becomes empty again). This was chosen so that the FIFO does not fill up over time if the UART transmitter cannot keep up and practically means that the FIFO will only ever contain a maximum of one measurement at any given time. While not the most conventional use of a FIFO, it allows for the DDC integration time to be independent of the data transmission speed: the shortest integration times are still available as the time taken for a measurement to be stored into the FIFO is negligible compared to the UART transmission rate. Note that a 32-bit custom end-of-line sequence is added to the end of each measurement, used to separate measurements when saved to a text file on a PC. Since each byte sent over UART contains a start and stop bit, there are actually 640 + 32 + 84(2) = 840 bits to be sent over UART for the (640+32)/8 = 84 bytes of data saved in the FIFO per measurement.

Data sent over UART is saved to a text file using the CoolTerm serial terminal emulator. This program was chosen for its ability to view and save incoming data as hexadecimal (instead of ASCII), accept the 921600 baud rate and accept custom end-of-line strings such that each measurement is timestamped and saved to a new line in the text file, discarding the end-of-line string. While the application saves data, a C++ script is run that reads the latest measurement, converts it to charge values in pC and then plots the result live in a graph using ROOT. ASCII commands can also be sent from the PC to a UART receiver in the design, which is similar to the transmitter, to reset, pause and trigger acquisitions.

==Daisy-Chaining DDC232s==
DDCs can be easily daisy-chained for applications that require more inputs. A full clinical-ready range detector would require 160 scintillator sheets (10 daisy-chained DDC boards) to give a WET of approximately 450 mm – though the detector can offer any multiple of 16 sheets. To daisy-chain DDCs, CLK, CONV, DIN_CFG, CLK_CFG, RESET and DCLK are sent to all boards as usual, however DOUT of boards in the chain are connected to DIN of the board next along the chain. The last board in the chain has DIN connected to ground and DOUT of the first board is connected to the FPGA as normal. DVALID is cascaded through OR logic gates on the circuit boards to indicate that all DDCs are ready to shifted data out to the FPGA.

Adding more boards to a chain increases the number of data bits to be read out each cycle, which increases the integration time since data must be read out completely at least 10 μs before the next CONV toggle. Reading out data only before CONV toggles, the shortest integration time with one DDC is 181 μs. In order to daisy-chain 10 modules, the minimum integration time required is 470 μs. To increase the time available for shifting out data, one is able to read out data before and after CONV toggles, which gives the minimum integration time possible with 10 DDCs of 341 μs. The UART protocol limits data transfer speed with 10 DDCs to 0.71 KHz. The shortest possible integration time is chosen to be able to see fast changes in the light output of the scintillator as the dynamic range can always be reduced if the light output is too low. 

It is planned that the UART interface be upgraded to a full USB 2.0 interface, which will require use of the Zybo’s on-board ARM processor but could offer up to 480 times faster transfer speeds. Additionally, it is planned that data be transferred to a Raspberry Pi (Raspberry Pi Foundation, Cambridge, England) instead of a PC, for a compact solution which will host a web-page to allow live photodiode data to be viewed in any web browser at a refresh rate of 50Hz. A PC-based version of the live web-based plot will be developed first.

=Operation=
==Setting Up==
*The Zybo Z7 is powered with a 5V 2.5A power supply while the DDC232 board is powered with 12V 1.25A supply.
**Ensure the power supply jumper on the Zybo is set to WALL.
*Photodiodes are connected to the DDC232 with the star symbol on the bottom side (closer to the PMOD connections).
*The DDC232 is connected to PMOD header JC on the Zybo with jumper cables with a 1:1 connection except for pins 6 and 12 (GND pins) being connected to pins 5 and 11 on JC. This mismatch of the GND pins will likely be corrected in future DDC232 boards. 
**Pins 5 and 11 (3V3 pins) on the DDC232 need not be connected, as the DDC232 is powered with a PSU. Pins 6 and 12 on the Zybo JC are therefore empty. 
*A Digilent USB-UART PMOD adaptor is connected to pins 1-6 of PMOD header JD on the Zybo.
*Micro-USB to USB-A cables are connected from the USB-UART adaptor (for data transfer) and PROG UART on the Zybo (for configuration) to the PC.

==Configuring the FPGA==
*Opening the Vivado project containing the code to communicate with the DDC232, open up the hardware manager and press auto-connect. If the Zybo is connected and powered on, it should automatically be found. Then, press program device.
**The current debug core will be opened, but this can be ignored (unless you are debugging). 
**Ensure the RGB LED is green after programming, indicating correct configuration.
**Press BTN1 to pause operation for the time being.
*Close Vivado and reboot into MacOS and open CoolTerm.
**The USB cable connected to PROG UART can be removed.

==CoolTerm (UART)==
*Apply the following setting in CoolTerm options:
**Under Serial Port:
***Port: Select the port that the USB_UART adaptor is connected to.
***Baudrate: 921600
***Data bits: 8
***Parity: none
***Stop bits: 1
***Flow control: Leave boxes unchecked
**Under Receive:
***Receive buffer size: 84 (equal to the number of bytes per measurement including termination string)
***Capture format: Hexadecimal
***Leave "Format Hexadecimal Data" unchecked
***Check "Add timestamped to received data" with Type "Absolute time" and check "Wait for termination string", which is set to "0D 0A 0D 0A". Uncheck "retain termination string"
***All other boxes should be unchecked.
*Open the connection to the port.
*To begin capture, press CMD+R and save data as "capture.txt" to the same directory containing the writeCSV.cpp macro.
*Ensuring SW1 is set to 0 (continuous mode), press BTN0 to reset the FPGA and resume operation.
**Capture can be stopped by pressing SHIFT+CMD+R.
*To view incoming data on a local machine, you can use the ROOT macro plotLive.cpp, executed in the same directory that data is being saved: $ root -l "plotLive.cpp"
**Make the canvas smaller to increase the frame rate.
*Close the connection to the port before quitting the application.

==Uploading Data to HEP Server==
Data can be uploaded to the HEP cluster in order to run a webpage with live photodiode data.
*On your local machine:
**Move to the directory containing photodiode data being saved by CoolTerm and the C++ script to generate the CSV file: $ cd /Path/To/Data
**Generate the photodiode data with: $ root -l "writeCSV.cpp" 
**In a new terminal, move to the same directory and run the following command in order to copy this data, line by line, to the plus1 web area: $ tail -F -n 1 output.csv | ssh username@plus2.hep.ucl.ac.uk -T 'cat - > /unix/www/html/pbt/PDdisplay/photodiodeValuesAll.csv'
*On plus2 :
**In a new terminal, move into the directory where the photodiode data is written: $ cd /unix/www/html/pbt/PDdisplay/
**Run the following script in that directory: $ ./outputSingleLine.sh -v photodiodeValuesAll.csv photodiodeValues.csv 0.02
***This will output a single line CSV file - photodiodeValues.csv - every 10ms (100Hz). It's a good idea to set the write rate about 2x faster than the desired frame rate.
**To output this any more or less often, adjust the "waitTime" value at the end (needs to be in seconds).
**To stop displaying each line as it is written, remove the '-v' flag.
**Note that this script contains an infinite loop, so to stop it running you need to kill it by hand with Ctrl-C. You may need to do this a few times to get it to quit...

=Legacy Information=
The first version of the FPGA interface with the DDC232 used a Zybo Z7-10 and transferred data to a PC over the UART protocol. The board was eventually replaced with a Nexys Video for the larger FPGA and the FTDI chip to allow fast USB communication. Information specific to the Zybo is left here for reference.

==Zybo Z7-10==
Details of the design-assigned purpose of the on-board buttons, switches and LEDs are provided below:
*Clocking Wizard Status LED (LD0): the 125 MHz FPGA master clock is converted to a 120 MHz clock using the Xilinx Clocking Wizard intellectual property (IP), to allow for easier generation of the 10 and 20 MHz signals required by the DDC. This LED is on when the Clocking Wizard component of the design is generating a stable clock. All FPGA operations are timed using this clock.
*Test-mode LED (LD1): the DDC can be configured into a test diagnostic mode for debugging, in which all the inputs give a zero signal (slightly above zero due to noise and a negative current offset). This LED is on when the user has enabled the test mode, which can be toggled using the Toggle Test-Mode button (BTN3) and becomes active after a Global Reset press.
*State LED 1 (LD2): this LED is on whenever the DDC in the power-up, idle, or configuration state.
*State LED 2 (LD3): this LED is on whenever the DDC is measuring or shifting out data.
*Acquisition mode (SW0): this switch toggles between the continuous and triggered acquisition modes. In continuous mode, data will be output to the PC as fast as possible, the rate of which is dependent on the chosen integration time and the speed of the data transfer. When in triggered mode, the next measurement after the Trigger Acquisition button (BTN2) is pressed is sent to the PC.
*FSR Switches (SW1-3): these 3 switches control the 3-bit FSR code, which allows the user to choose between the 8 different dynamic ranges.
*Global Reset (BTN0): this button resets all aspects of the FPGA and DDC. Any changes since the last reset in the FSR, acquisition mode or test-mode settings are applied. The DDC returns to its power-up state and reconfigures before resuming operation.
*Pause Operation (BTN1): this button idles the FPGA and DDC, all signals are held at their default value apart from the 120 MHz master clock. Pressing Global Reset restarts operation.
*RGB LED (LD6): this LED will be green if the DDC is configured correctly, red if configuration is incorrect or if too short an integration time is chosen, or blue if the first-in first-out (FIFO) interface is read when empty or is written to when full.

==Setting Up==
*The Zybo Z7 is powered with a 5V 2.5A power supply – ensure the power supply jumper on the Zybo is set to WALL.
*The DDC232 is connected to PMOD header JC on the Zybo with jumper cables with a 1:1 connection on the 2nd DDC232 prototype. On the first DDC232 prototype pins 6 and 12 (GND pins) are connected to pins 5 and 11 on JC.
**Pins 5 and 11 (3V3 pins) on the DDC232 need not be connected, as the DDC232 is powered with a PSU. Pins 6 and 12 on JC are therefore disconnected. 
*A Digilent USB-UART PMOD adaptor is connected to pins 1-6 of PMOD header JD on the Zybo.
*Micro-USB to USB-A cables are connected from the USB-UART adaptor (for data transfer) and PROG UART on the Zybo (for configuration) to the PC.