Configuring and monitoring link firmware

The sections below outline the empbutler commands for manually configuring and testing the link firmware, for both loopback and board-to-board tests. As part of these tests, the datapath firmware is configured to “play” pattern data out of the TX (transmit) MGTs into the RX (receive) MGTs, and then capture the results.

Note

In these commands, you will need to replace the following arguments with appropriate values:

• CONNECTIONS_FILE.xml : Relative/absolute path to your connections file

• DEVICE_ID : ID string for your FPGA in that connections file

• RX_CHANNEL_LIST / TX_CHANNEL_LIST / CHANNEL_LIST: Comma-separated list of input/output channels (e.g. 4-7,12 for links 4, 5, 6, 7 and 12)

Loopback

1. Add the URI and address table for your device to a uHAL connections file (if not done already)

2. Setup your environment

   export PATH=/opt/cactus/bin/emp:$PATH
   export LD_LIBRARY_PATH=/opt/cactus/lib:$LD_LIBRARY_PATH
   

3. Reset the TTC block

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID reset internal
   

4. Load pattern into the output channels’ buffers, and configure those buffers to send that data into the TX MGTs at the start of each LHC orbit

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID buffers tx PlayOnce -c CHANNEL_LIST --inject generate:counter-with-index
   

   Note: The PlayOnce argument here specifies that the data from the input file should only be played into the TX MGTs at the start of each LHC orbit (i.e. once per 3564 cycles of the LHC clock). If this argument is changed to PlayLoop, then the data from the input file will be played continually in a loop (i.e. the data played into the TX MGTs on the Nth clock cycle will be N % 1024).

5. Reset and configure the link firmware

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID mgts configure tx -c CHANNEL_LIST --loopback=nearPMA
   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID mgts configure rx -c CHANNEL_LIST
   

   By default, these commands will automatically check the status of the link firmware. The status check can be skipped by adding the --no-check flag to the end of each configure command, or run separately using the mgts status command:

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID mgts status -c CHANNEL_LIST
   

6. Align data received on each channel, so that the data valid signal rises from 0 to 1 on the same clock edge for all channels.

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID mgts align -c CHANNEL_LIST
   

7. Configure input channels’ buffers to capture data received from RX MGTs, then download the data from the input and output buffers.

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID buffers rx Capture -c CHANNEL_LIST
   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID capture --rx CHANNEL_LIST --tx CHANNEL_LIST
   

   By default, the captured rx/tx data is written to data/rx_summary.txt and data/tx_summary.txt, but that directory can be changed using the -o argument. If you open up the RX capture file - e.g. less -S data/rx_summary.txt (the -S flag prevents line wrapping) - then you should see the input data after 20 to 30 clock cycles, with the first valid word for each channel occurring on the same clock cycle.

Board-to-board tests

On each board:

1. Add the URI and address table for your device to a uHAL connections file (if not done already)

2. Setup your environment

   export PATH=/opt/cactus/bin/emp:$PATH
   export LD_LIBRARY_PATH=/opt/cactus/lib:$LD_LIBRARY_PATH
   

3. Reset the TTC block, configuring it to use an external source (if using an FC7 to distribute legacy-style TTC signals, change tcds2 to legacy here)

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID reset tcds2
   

Then, on the board that will be transmitting data (or on both boards if you want bidirectional data transfer):

1. Load a pattern into the output channels’ buffers, and configure those buffers to send that data into the TX MGTs at the start of each LHC orbit

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID buffers tx PlayOnce -c TX_CHANNEL_LIST --inject generate:counter-with-index
   

   Note: The PlayOnce argument here specifies that the data from the input file should only be played into the TX MGTs at the start of each LHC orbit (i.e. once per 3564 cycles of the LHC clock). If this argument is changed to PlayLoop, then the data from the input file will be played continually in a loop (i.e. the data played into the TX MGTs on the Nth clock cycle will be N % 1024).

2. Reset and configure the TX link firmware

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID mgts configure tx -c TX_CHANNEL_LIST
   

   Add the --invert flag to the end of the command if the TX polarity needs to be inverted; the command can be run multiple times for different sets of channels, if inversion is only required for some of them.

Finally, on the board that will be receiving the data (or on both boards if you want bidirectional data transfer):

1. Reset and configure the RX link firmware

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID mgts configure rx -c RX_CHANNEL_LIST
   

   Add the --invert flag to the end of the command if the RX polarity needs to be inverted, or the --enable-dfe flag if you want to enable DFE mode. The command can be run multiple times for different sets of channels if inversion/DFE is only required for some of them.

   By default, this command will automatically check the status of the link firmware. The status check can be skipped by adding the --no-check flag to the end of each configure command, or run separately using the mgts status command:

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID mgts status rx -c CHANNEL_LIST
   

2. Align data received on each channel, so that the data valid signal rises from 0 to 1 on the same clock edge for all channels.

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID mgts align -c RX_CHANNEL_LIST
   

3. Configure input channels’ buffers to capture data received from RX MGTs, then download the data from the input buffers.

   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID buffers rx Capture -c RX_CHANNEL_LIST
   empbutler -c CONNECTIONS_FILE.xml do DEVICE_ID capture --rx RX_CHANNEL_LIST
   

   By default, the captured rx/tx data is written to data/rx_summary.txt and data/tx_summary.txt, but that directory can be changed using the -o argument. If you open up the RX capture file - e.g. less -S data/rx_summary.txt (the -S flag prevents line wrapping) - then you should see the input data after 20 to 30 clock cycles, with the first valid word for each channel occurring on the same clock cycle.