Advanced Usage¶

[edit this page]

The following is an in-depth list of commmands that cover more advanced configuration and usage of the hardware on the SERENITY boards including clock jitter cleaners and the Fireflys transceivers. For basic usage of the SERENITY it is suggested to start here: SERENITY first steps. Some more basic SMASH are also covered within the Useful Commands section.

SMASH employs regex for taking wildcards and expansions of commands.

Note that commands which contain spaces (e.g. Foreach type LTM.* "Store to NVM") must be delimited by double quotes. The tab-completion feature will auto-complete commands, but without the first double quote the command will not be executed correctly.

Set up the local environment¶

export LD_LIBRARY_PATH=/opt/cactus/lib:/opt/smash/lib:$LD_LIBRARY_PATH
export SMASH_PATH=/opt/smash
export PATH=/opt/smash/bin:$PATH

In the following commands a configuration/settings file is required in order to define the actual hardware board that is being interrogated/controlled via SMASH - someBoardConfig.smash. This file can define the board configuration (e.g. bindings between components and sockets) as well as settings (e.g. clock or firefly set-up). The directory /opt/smash/etc/serenity should be investigated in order to see which configurations are available and choose the appropriate one. Alternatively all the configurations could be loaded sequentially if you want the maximum options (testing purposes):

Apply a fixed configuration file (obviates the -c filename requirement):

export SMASH_DEFAULT_CONFIG=/path/to/someBoardConfig.smash

Run interactively:

smash.exe -i

or

smash.exe -c /path/to/someBoardConfig.smash -i

Run commands on the command-line (list of quoted commands can be extended):

smash.exe -c /path/to/someBoardConfig.smash "my command1" "my command2"

or:

smash.exe "my command1" "my command2"

if the SMASH_DEFAULT_CONFIG environment variable has been set.

Sometimes the ARTIX FPGA is not recognised on the PCIe bus therefore the pcie_reconnect script needs to be run at least once:

sudo pcie_reconnect_serenity.sh
If your path is not recognised by sudo then:
sudo /opt/smash/bin/pcie_reconnect_serenity.sh

Detailed Commands¶

A number of simple commands are provided here within the Useful Commands section. Here are described more elaborate or complicated functionality of SMASH.

Power On or Off both of the processing FPGAs¶

Note that different daughter cards require slightly different voltages for operation. Automatic reconfiguration of the power supplies depending on the daughter card type is underway, but until it is complete do not swap daughter cards between boards unless you an expert and know how to reconfigure the power supplies.

smash.exe -c /path/to/someBoardConfig.smash "X0:Power On"
smash.exe -c /path/to/someBoardConfig.smash "X0:Power Off"

To power on a given site:

From the commandline:

script option:

smash.exe -c /opt/smash/etc/serenity/configuration/[Your board configuration] "run /opt/smash/etc/serenity/examples/power_x0_on.smash"

Embedded command:

smash.exe -c /opt/smash/etc/serenity/configuration/[Your board configuration] “X0:Power On”

Interactive mode:

Run /opt/smash/etc/serenity/examples/power_x0_on.smash

Or the much simpler:
```
X0:Power On
```

To power off a given site:

From the commandline:

smash.exe -c /opt/smash/etc/serenity/configuration/[Your board configuration] "Run /opt/smash/etc/serenity/examples/power_x0_off.smash"

Or

smash.exe -c /opt/smash/etc/serenity/configuration/[Your board configuration] “X0:Power Off”

Or in interactive mode:

Run /opt/smash/etc/serenity/examples/power_x0_off.smash

Or the much simpler:

X0:Power Off

To power on both sites:

From the commandline:

smash.exe -c /opt/smash/etc/serenity/configuration/[Your board configuration] “X0:Power On” “X1:Power On”

Or:

smash.exe -c /opt/smash/etc/serenity/configuration/[Your board configuration] “Foreach name X.:Power On”

In interactive mode:
```
Foreach name X.:Power On
```

To power off both sites:

From the commandline:

smash.exe -c /opt/smash/etc/serenity/configuration/[Your board configuration] “X0:Power Off” “X1:Power Off”

Or:

smash.exe -c /opt/smash/etc/serenity/configuration/[Your board configuration] “Foreach name X.:Power Off”

In interactive mode:
```
Foreach name X.:Power Off
```

Configure both the SI5345 clock jitter cleaners on an interposer¶

Allows configuration of the clock jitter cleaners with a text file generated the SiLabs ClockBuilder Pro software.

smash.exe -c /path/to/someBoardConfig.smash "Foreach type Si5345 Configure /opt/smash/etc/serenity/clocks/Si5345-RevB-LHCP100F-100MHz-LHC-Precision-Clock-Registers-v2.txt

where the filename is just one example which maybe packaged within the smash RPM.

The Clock Pro software for configuring the clocks using I2C can be downloaded from the SiLabs-website (windows only) and the appropriate chip is a B-part.

The directory /opt/smash/etc/serenity/clocks/ contains a number of pre-built clock settings file. The correct one should be chosen for the desired clock configuration.

More information about the clock distribution network available on the SERENITY is described here.

Status Checks¶

The LTM4677 PSUs supply the FPGA transceivers (~1V), optical parts (1.8V) & general I/O (1.8V). The NDM3Zs supply the FPGA core voltage (~1V) and optical parts (3.3V)

smash.exe -c /path/to/someBoardConfig.smash "Foreach type LTM.* Measure .*"
smash.exe -c /path/to/someBoardConfig.smash "Foreach type NDM.* Measure .*"
smash.exe -c /path/to/someBoardConfig.smash "Foreach type .* Validate"

The last command essentially scans the I2CBus to give the result of all connected devices.

Apply multiple configuration files within one directory (alphanumeric order):¶

smash -c "[some_path]/*.smash"

Note that you need the quotes to prevent it being shell-expanded on the commandline. However, the scripts directory is deliberately configured to separate the “base board” from the “configuration”. The configuration scripts explicitly instantiate the bare board in each case and then create and plug a daughtercard and any fireflys. You will find a folder called “configurions” which contain such useful scripts as “BareBlade”, “RalBoard”, “TomOpticalTests” and “PizzaBox”. At least two of these scripts also create and configure the XVC on behalf of the end user.

To connect a DC to the JTAG header:¶

From the commandline:

smash.exe -c /opt/smash/etc/serenity/configuration/[Your board configuration] 'Run /opt/smash/etc/serenity/examples/jtag_header_x0.smash'

Or in interactive mode:

Run /opt/smash/etc/serenity/examples/jtag_header_x0.smash

Or explicitly:

JTAGmux Select JTAGheader
DC0:FPGA JtagManualConnect
DC1:FPGA JtagManualConnect

The preferred method of connection is to use an XVC (Xilinx Virtual Cable) rather than using a JTAG interface. Instructions for setting up the XVC are described in XVC (Xilinx Virtual Cable). The configuration to setup SMASH for XVC is described below.

To connect a DC to the XVC:

In interactive mode:

JTAGmux Select IPbusJTAG
DC0:FPGA Decorate XVC 5000

To connect the other DC replace DC0 with DC1. It is no longer possible to daisy-chain the XVC. The XVC decoration should print to the screen a series of TCL instructions which can be run in Vivado to activate the XVC for programming of the FPGA. In the case of port forwarding (tunnelling) the IP address has to be replaced with localhost on the Vivado (hw_server) machine.

If you want the full JTAG chain printed out

XVC Print

Tunnel to run XVC through a port forward¶

Tunnel to run XVC through a port forward (server PC) onto a local subnet of 10.0.0.XX:

JTAGmux Select IPbusJTAG
DC0:FPGA Decorate XVC 5000

In order to open up the 5000 port through a tunnel the following command is required:

ssh -L 5000:localhost:5000 -p 222 cmx@pcuptracker001.cern.ch

Note that in the case of port forwarding then the command open_hw_target -xvc_url 128.141.YYY.XXX:5000 given by SMASH needs to be modified such that 128.141.YYY.XXX:5000 instead says localhost:5000.

Voltage and Current Measurements¶

To measure voltages on the LTMs and NDMs:

In interactive mode:

Foreach type LTM.* Measure Vout.*
Foreach type NDM.* Measure Vout.*

To measure currents on the LTMs and NDMs:

In interactive mode:

Foreach type LTM.* Measure Iout.*
Foreach type NDM.* Measure Iout.*

To measure input voltages on the LTMs and NDMs:

In interactive mode:

Foreach type LTM.* Measure Vin
Foreach type NDM.* Measure Vin

Example Configuration File:¶

TomOpticalTests.smash:

Run /opt/smash/etc/serenity/base/atca/*.smash

Run ../DaughterCards/KIT_KU15P.smash DC1
Plug I1 DC1 # Socket - Module ordering

Create FireflyBi FF1

Plug J1:19 FF1 # Socket - Module ordering

# By default we want to route all JTAG via IPbus
JTAGmux Select IPbusJTAG

DC1:FPGA Decorate XVC 5000

The Create FireflyBi FF1 adds the physical firefly to the element list.
The Plug J1:19 FF1 binds FF1 to the socket J1:19.

Locate Command:¶

If you wish to see where a physical component is on the SERENITY board then the very handy Locate command can be run:

Locate FF1
Locate J1:19

Both should give the same answer.

Firefly Configuration:¶

The different types of Firefly: bidirectional, passive, tx & rx all have different funtionality within SMASH.

For the Bi’s, the functions are:

"Tx Transmit" [Channel expression] [Enable|Disable]
"Rx Output" [Channel expression] [Enable|Disable]
"Tx CDR" [Channel expression] [Enable|Disable]
"Rx CDR" [Channel expression] [Enable|Disable]

"Tx Squelch" [Channel expression] [Enable|Disable]
"Rx Squelch" [Channel expression] [Enable|Disable]

Or "Foreach type FireflyBi"

"Set Tx CDR rate" [Channel expression] [Hi|Lo]
"Set Rx CDR rate" [Channel expression] [Hi|Lo]
"Set Tx Polarity" [Channel expression] [Normal|Invert]
"Set Rx Polarity" [Channel expression] [Normal|Invert]
"Set Tx Equalization" [Channel expression] [9.8dB|8.8dB|8.2dB|7.2dB|6.5dB|4.8dB|3.7dB|2.7dB|1.9dB|1.3dB]
"Set Rx Pre-emphasis" [Channel expression] [7.5dB|6dB|5dB|4dB|3dB|2dB|1dB|0dB]
"Set Rx Amplitude" [Channel expression]  [300mV|450mV|600mV|900mV]

For unidirectionals there is a lot less configurability

Tx commands are:

"Soft Reset"
"Channel" [Channel expression] [Enable|Disable]
"Set Polarity" [Channel expression] [Normal|Invert]

Rx, the commands are:

"Soft Reset"
"Channel" [Channel expression] [Enable|Disable]
"Output" [Channel expression] [Enable|Disable]
"Set Amplitude" [Channel expression] [Off|Low|Medium|High]
"Set De-emphasis" [Channel expression] [Off|On]

Then it is possible to return the I2C information from the Fireflys:

Foreach type FireflyBi.* Validate
Foreach type FireflyRx.* Validate
Foreach type FireflyTx.* Validate

"Foreach type Firefly.* Validate"

Note the incolusion of a dot before the *, if you instead used Firefly* this would match Firefl, Firefly, Fireflyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy but not FireflyBi, FireflyRx or FireflyTx

Firefly Hard Reset:¶

ForEach type Firefly.* "Hard Reset"

biFF2n "Hard Reset"

Firefly High-Speed¶

If you want to run the FireflyBi’s at 25.5 - 26.0 GHz then CDR must be set Lo
```
Foreach type FireflyBi Set Tx CDR rate [Channel expression] Lo
```
If you want to run the FireflyBi’s at 27.5 - 28.05 GHz then CDR must be set Hi
```
Foreach type FireflyBi Set Tx CDR rate [Channel expression] Hi
```

Anything outside these ranges may not work unless CDR is off.

If the FPGA or whole board are power-cycled then a hard reset is requires of the Fireflys:

ForEach type Firefly.* "Hard Reset"

List Available Options in SMASH¶

[element name] List [Functions|Ports|Measurements|Attributes]

e.g.

U:6 List Functions

Will give the following output:

SetAttribute
List
Schedule
Measure
Print
Validate

Or for the Si5345 clock jitter cleaners:

U1:3 List Functions

will give:

Disable
SetAttribute
Enable
List
Schedule
Configure
Measure
Print
Validate