where B1 and B2 are boolean expressions which are the comparators in this scheme. R1, R2, and R3 are expressions which evaluate to 2-bit values that determine which of the four output ports a packet is sent out. The comparators are defined as follows.
B2 = (DOWNROUTE and RULE2_MASK) equal? RULE2_REF
Where RULE1_MASK, RULE2_MASK, RULE1_REF, and RULE2_REF are all defined in Arctic's configuration registers. B1 and B2 are used to determine which of R1, R2, or R3 is used to determine the output port.
The reader may note that this routing scheme is rather complicated, and we have not even discussed the definitions of R1, R2, and R3! However, we can already see an opportunity to try a simple test. By setting both RULE1_MASK and RULE1_REF to 0, all packets sent into the system should emerge from the port determined by R1. This is the simple test performed by test003 in our system.
00000001
0ffbdff8
040f7bfe
00000000
00000000
000300ff
Figure A-1: test003.config file
The first step in defining this test is to create the test003.config file, which is shown in Figure A-1. The file consists of only numerical data, as Verilog requires, and the numbers shown correspond to the actual values loaded into the configuration registers. Each of arctic's input ports has five 32-bit configuration registers, and one 32-bit datum is given for each of these registers. The 00000001 that appears at the top indicates that these values should be copied into the configuration registers of every arctic input port. If this value were a 00000000, then five 32-bit values could be listed for each of the four input ports. I will not detail the meaning of each bit in the configuration information. Suffice it to say that RULE1_MASK, RULE2_MASK, RULE1_REF, and RULE2_REF are all set to 0.
start_sequence("../sequences/test003/test003.config",
"../sequences/test003/test003.log.pkts",
"../sequences/test003/test003.log.stats",
"../sequences/test003/test003.log.errs",`SPEED,`TIMED);
in.send_packets("../sequences/test003/test003.a.pkts");
lclk_delay_cycles(100);
end_sequence;
Figure A-2: test003.code file
Now that the initial configuration has been specified, the list of actions needs to be specified in test003.code, which appears in Figure A-2. The only action taken in this test group is the sending of a set of packets, and, as a result, this file is very simple. The first line carries out the configure phase of the test group. (The function is called start_sequence because test groups were originally called test sequences.) The arguments to start_sequence are the file names of the log files that will be output at the end of the test, plus the SPEED argument mentioned in Section 4.3, plus another flag that we shall discuss in Section A.5. Following this action is a send_packets command and an lclk_delay_cycles command that makes the system wait for 100 local clock cycles before entering the check phase. The final command carries out the check phase. This is all that is needed for a very simple test such as this, and this file is about as simple as a .code file gets.
00000000000000008
00000000040081080
00000000040081082
00000000040081084
00000000040081086
00000005040081f80
00000005040081f82
00000005040081f84
00000005040081f86
Figure A-3: test003.a.pkts file
Next we need to specify the packets that are going to be inserted into the system. Commands that will direct the system to insert a packet will be kept in a file named test003.a.pkts (shown in Figure A-3), which is the argument to the procedure send_packets. For this test, test003.a.pkts contains only eight commands, one for each packet inserted into Arctic. The value at the beginning of the file is the number of packet send commands contained in the file, and it is followed by the commands themselves. Each of these commands consists of three fields. The left most field is a 32-bit number specifying the number of cycles after calling send_packets that the transmission of the packet should begin. The middle field is the 32-bit packet identifier, and the last four bits specify which input port the packet should be sent to (shifted left by one bit). In this packet insertion file, the first half of the commands insert packet 04008108, and the last half insert packet 040081f8. The first four are sent to each of the four input ports at cycle 0, and the next four are sent to each input port at cycle 5.
040081081000000000
040081081000000000
040081081000000000
040081081000000000
040081f81000000000
040081f81000000000
040081f81000000000
040081f81000000000
1f0000000000000000
Figure A-4: test003.chk.pkts file
With all the inputs to the system specified, all that remains is to create the check files which will determine what the generated log files should look like. The file test003.chk.pkts appears in Figure A-4 and consists of eight lines specifying the packets that should come out of Arctic during the test followed by a unique value, 1f0000000000000000, that marks the end of the file. The lines that specify how packets should be received are made up of four fields. The left most field is the packet identifier, and it is followed by a four-bit field specifying the output port the packet was sent from, and another four bit field which is 1 if every bit in the received packet should match every bit of the sent packet. The right most 32-bit field corresponds to the time that the packet is sent out, but we will ignore this field until Section A-6. In this file, we have one entry for each packet, each of which should be received from output port 1, and none of which should match the transmitted packet bit-for-bit, because a new checksum word should be inserted at the end.
000000000
Figure A-5: test003.chk.stats file
00000000000000000
Figure A-6: test003.chk.errs file
The files test003.chk.stats and test003.chk.errs appear in Figures A-5 and A-6 respectively. Both consist of nothing but trailer words of all zeros that signal the end of the files, because during this test, neither the statistics information nor the errors and control information was ever read from Arctic.
We have completed the last step in specifying this test. The test exercises a specific part of the routing logic, and at the same time gives a nice, basic test to see if the packet transmission mechanisms are working correctly. The reader may note that specifying this test was a very involved and time-consuming process, because of both the complexity of Arctic and the cryptic format of the files. The time needed to create a test can be reduced by duplicating and modifying old tests to make new ones, rather than creating new ones from scratch, but this is still rather complicated because the test is spread over so many files. In Section 6.6, I explore just how much of a burden this complexity is.
start_sequence("../sequences/test060/test060.config",
"../sequences/test060/test060.log.pkts",
"../sequences/test060/test060.log.stats",
"../sequences/test060/test060.log.errs",`SPEED,`TIMED);
mn.clear_stats(`SPEED); //clear statistics
in.send_packets("../sequences/test060/test060.a.pkts");
lclk_delay_cycles(100);
write_stats(0,`SPEED);
write_stats(1,`SPEED);
end_sequence;
Figure A-7: test060.code file
Let's take a briefer look at test060, which runs some basic tests of Arctic's statistic's counters. The file test060.code is shown in Figure A-7. After the initial start_sequence command, the statistics are cleared with the command clear_stats. This needs to be done first because resetting Arctic does not clear the statistics counters. After this is done, a set of eight packets is sent into Arctic, and the system waits 100 cycles before starting to read out statistics information. The statistics are then read out twice, first non-destructively, and then destructively (this is determined based on the value of the first argument to write_stats). A destructive read clears a statistics counter when it is read, so in this test, the first set of statistics should be nearly identical to the second.
000000001
000000000
000000000
000000000
000000000
000000xxx
000000000
000000000
000000000
000000000
000000000
000000xxx
000000000
000000004
000000000
000000000
000000004
000000xxx
000000000
000000004
000000000
000000000
000000004
000000xxx
000000000
<All the above text is repeated here>
000000000
Figure A-8: test060.chk.stats file
The file test060.chk.stats appears (in part) in Figure A-8. Since statistics information is read out twice, this file contains two reports of the statistics information (though only one is shown because the second is exactly the same as the first). Each report contains a header word, 000000001, followed by each of the 24 statistics values, arranged in order of increasing address, as specified in the Arctic User's Manual [5]. The statistics specified as `` x'' are all idle statistics that increment once for every three cycles that an output port is not busy or waiting. We put an `` x'' in these locations because we are not interested in these statistics during this test. At the end of the file is a trailer word of all zeros, as in the previous test group.
Each of the statistics locations must be read independently through Arctic's maintenance interface. Even ignoring configuration time, this test group can take over an hour to run, even on the fastest computers this group has available. Fortunately, if someone is trying to debug the statistics counters, it is possible to run the test in quick mode, in which case the test will run in about five minutes. For this test, quick mode is even more helpful than usual.
start_sequence("../sequences/test052/test052.config",
"../sequences/test052/test052.log.pkts",
"../sequences/test052/test052.log.stats",
"../sequences/test052/test052.log.errs",`SPEED,`TIMED);
in.send_packets("../sequences/test052/test052.a.pkts");
begin
#12 force data_ni3[0] =1; // this should cause an idle error
#10 release data_ni3[0]; // and a freeze before either of two
end // packets arrive on input 3
begin
#352 force data_ni2[0] = 1; // This should cause a length error on
#10 release data_ni2[0]; // input 2 (but no checksum error
end // because the checksum of this packet
// assumes this force). A freeze should
// occur before input 2 receives a
// second packet. The second packet is
// addressed to output 2.
// Packet with bad checksum is being inserted on input 1.
// It should cause input 1 to get a checksum error and
// freeze (before getting a second packet and the second
// packet is addressed to output 2).
begin
#382 force frame_ni0 = 1; // This should cause a 11 pattern where a
#10 release frame_ni0; // 10 is expected. That in turn should
end // cause a frame error on input 0. A
// freeze should occur before input 0
// receives a second packet and the
// second packet is addressed for output 2
join
#2000 write_errs_contr(0,`SPEED);
end_sequence;
Figure A-9: test052.code file
The first error is generated when a bad value is forced on input port 3's data transmission line at a moment when no packet is being transmitted. This should cause Arctic to increment port 3's idle error count. Next, a value is forced on input port 2's data line at the moment a packet's length field is being transmitted. This should cause Arctic to increment the length error counter for port 2. Finally, the frame wire for input port 0 is given a bad bit, so that the frame error counter for port 0 will increment.
00000000000000001
00000000000010000
00000000000000001
00000000001000000
00000000100000000
11000000000000000
00000000000000000
xxxxxxxxxxxxxxxxx
00000000000000000
Figure A-10: test052.chk.errs file
By the end of this test, four errors should have been detected (the above three plus a checksum error because one of the packets transmitted had a bad checksum word). The file test052.chk.errs in Figure A-10 reflects these errors. This file consists of one report of the errors, control, and free buffer information, which is generated by the line write_errs_contr(0,`SPEED) in the file test052.code. The `` 0'' argument indicates that the error counters should be read non-destructively, and `` `SPEED'' is the familiar quick mode flag. Each report contains 8 binary words. The first is simply a header. The next four are the error counters for each port, in order, with the bits arranged as specified in the Arctic User's Manual [5]. These are followed by the first half and then the last half of the value stored in Arctic's control register. The last line contains the free buffer counters, also encoded as specified in the Arctic User's Manual. In this file, the final line consists only of `` x'' entries, because free buffer information is not important to the test. As before, the last line of the file consists entirely of zeros.
Tests like this one are very precise, but they do take a very long time to generate. This is one of the prime motivations for the creation of a randomized tester presented in Chapter 5.
These functions are important, but they are not commonly used during normal operation of the chip. Building complex functions into the functional testing system to test these functions, therefore, seems a bit wasteful. Unfortunately, they are hard to test by brute force using Verilog commands in a .code file, because they require coordination between at least 175 of Arctic's pins. For this reason, we compromised our testing standards for JTAG functions, and wrote special procedures that execute them, but do not verify that they complete correctly. This way, the user can let the system worry about executing the JTAG functions, but still needs to watch the output pins of Arctic to verify that the functions are working properly.
As an example, the extest(vector) procedure can be placed in a .code file, and cause the system to carry out an Extest function with the given vector. The vector has one bit for each register in the boundary scan ring and is split up, at the beginning of the function, into bits corresponding to input pins and bits corresponding to output pins. The next step is to force the input values onto Arctic's input wires and begin the Extest function by capturing input values. Then, a vector that contains the appropriate output bits is scanned into Arctic, and at the end of the test, the user must verify that the correct input bits were captured and scanned out of Arctic, and the correct output bits are being output from the appropriate output pins. This seems like quite a task for the user, but it can be done fairly easily for some simple input patterns, such as all 0's, 1's, or checkerboard patterns.
Some kind of randomized generation of packets is needed to take this burden off the user and guarantee that a wide variety of packets are being sent through the system. Our first attempt to solve this problem was to create programs that generated packet libraries and packet insertion files randomly. This works well up to a point, but since it is very hard to determine exactly when packets will leave Arctic without running a simulation, it is very hard to predict when buffers will free so that new packets can be transmitted to Arctic. Remember that each packet insertion command must specify exactly when each packet is transmitted.
To solve this problem, we introduced another mode of operation to the testing system, randomized insertion mode. In this mode, each stub keeps a count of the number of free buffers in the input port it is connected to, just as an Arctic output port would. With this ability, the stubs can handle flow control themselves and send packets as fast as possible. This mode is chosen with the last argument to start_sequence, which is TIMED if the original insertion method is used, and PACKED if this newer method is used.
start_sequence("../sequences/test032/test032.config",
"../sequences/test032/test032.log.pkts",
"../sequences/test032/test032.log.stats",
"../sequences/test032/test032.log.errs",`SPEED,`PACKED);
in.send_packets("../sequences/test032/test032.a.pkts");
lclk_delay_cycles(2450);
end_sequence;
Figure A-11: test032.code file
Test032 is the only test we created which uses this mode, and the file test032.code appears in Figure A-11. A library of random packets and a random packet insertion file were generated with utility programs. This was a useful test because it sent a large number of widely varied packets through Arctic, but even before it was completed, we realized that it was still very cumbersome and did not greatly simplify test generation in general. The use of packet library and insertion file generation programs was awkward and did not allow us to generate an unlimited number of random tests. This was another motivation for building the random tester described in Chapter 5.
Up to now, we have not discussed how this functional testing system checks the timestamp for each received packet recorded in the testxxx.log.pkts files. This is because there were no facilities for checking this field until we addressed this problem. We decided that each received packet should be given a rank in the check file that would determine when the packet should arrive. This rank would be stored in the part of the received packet check file that corresponded to the time of receipt in the log file. Instead of checking the exact time of receipt, the system would check that all packets of rank 1 were received before any packet of rank 2 and so on. This would make it possible to determine the order of packets, either individually or for very large groups. In addition, a rank of 0 would indicate that a packet could be received at any time, giving the user the ability to mark some packets ``don't care'' if the order of these packets was unimportant.
These ranks were used in test030 to test the ordering of packets. In this test, 80 packets were sent into Arctic in 10 groups of 8. The first group was given rank 1, the second group rank 2, and so on, as can be seen in the abbreviated version of the test030.chk.pkts file in Figure A-12. While it was not the case that all packets had to emerge exactly in order, each packet of one group had to be received before any packet of another group. If the test did not generate an error, then it could be said that Arctic was working well. If some packets did leave Arctic out of order and cause an error, however, the send times and receive times of each of these packets would have to be checked to make sure that they did not stay in Arctic for an ``unreasonable'' amount of time.
048111e80100000001
048111e80100000001
048111e80100000001
048111e80100000001
048111e80100000001
048111e80100000001
048111e80100000001
048111e80100000001
048111e80100000002
048111e80100000002
048111e80100000002
048111e80100000002
048111e80100000002
048111e80100000002
048111e80100000002
048111e80100000002
.
.
.
e48111e87100000008
e48111e87100000008
e48111e87100000008
e48111e87100000008
e48111e87100000008
e48111e87100000008
e48111e87100000008
e48111e87100000008
e40111e87100000009
e40111e87100000009
e40111e87100000009
e40111e87100000009
e40111e87100000009
e40111e87100000009
e40111e87100000009
e40111e87100000009
1f0000000000000000
Figure A-12: test030.chk.pkts file
This was imprecise solution to an imprecise problem. The test was not by any means guaranteed to find every bug in the output ports' scheduling algorithm, but it did give some idea of how well the system was working. This was a very troublesome test, however, because every packet that was a potential problem had to be tracked down. Fortunately, the specifications for Arctic were later changed, and packets delivered from any point to any other point were guaranteed to be kept in first-in-first-out order, which is a much easier ordering to test. We decided to implement routine checks for this ordering in our randomized tester.
This long list of tests has been included to give some impression of the huge amount of effort it takes to test a chip as large and complex as Arctic. The wide variety of features necessitates a very flexible system, which should help to explain why the user is given such precise control over this system. For those readers desiring even more detail about Arctic's testing system, Appendix B contains the system's user's manual, which presents every detail about the system that the user could possibly need.