Hardware timestamping

Conventionally, each packet or cell is given a timestamp by the host machine's kernel (i.e. in software) when the kernel driver is notified that a new packet has arrived. This approach results in poor quality timestamps for several reasons, among them the considerable latency and jitter between the packet arriving at the network interface and receipt by the kernel driver and uncertainty caused by interrupt coalescing wherein one host interrupt signifies the arrival of several packets. Such poor quality limits what research can usefully be done on network performance and related fields.

To solve this, the DAG generates timestamps in the hardware as close to the network interface as possible. Not only does this obviate latency, jitter and problems caused by interrupt coalescing, the hardware is capable of much greater accuracy and precision than software-generated timestamps. Precision comes from the freedom of custom hardware to assign as many bits to the timestamp as required and accuracy is assured by reference to an external time source such as GPS which is accurate to ± 40 nanoseconds.^[8] In contrast, the accuracy of NTP (by which kernel clocks can be corrected over the Internet) is in the order of milliseconds (about 100,000 times less accurate), depending on the conditions involved.

The DAG produces 64 bit timestamps in fixed-point format with 32 fractional bits, giving a potential precision of ${\displaystyle 2^{-32}}$ seconds or 233 picoseconds. The actual precision offered varies with the particular model of DAG, the oldest giving 24 fractional bits (60 nanoseconds) and better precisions offered in DAGs for higher bandwidth networks.^[9]

The timestamp is derived from a free-running clock provided by a crystal oscillator but the accuracy of crystals drift with both temperature and age. The DAG's solution is to use direct digital synthesis using the 1 Hz pulse-per-second output that many GPS receivers provide as its reference clock. This mechanism is described in §5.5.3 of Stephen Donnelly's PhD thesis^[10] which also describes in detail the pre-commercial era models of DAG.

Crucially, and an academically significant contribution of the DAG, the ability to use an external reference such as globally synchronised GPS makes it possible to do one-way time-of-flight measurements. This is of immense interest to academic researchers because packets flowing between two points on the Internet are neither guaranteed to follow the same path in each direction nor guaranteed to have the same timing characteristics in each direction.

Outside of the academic world, timestamp accuracy has commercial applications in the enforcement and compliance with law such as the EU Markets in Financial Instruments Directive 2004.

Packet loss

Almost as important as timestamp accuracy is guaranteeing 100% cell or packet capture and, where loss is unavoidable, knowing not only that packets have been lost but where. The "where" is important because, when analysing a packet trace, it's important to be able to compensate for lost packets when calculating inter-arrival times.

Most commercial NICs keep a count of dropped packets, but they can't indicate where packets were lost. The DAG prepends a header^[11] which, amongst other things, indicates how many packets were dropped between that packet and the previously accepted packet.

The DAG is also engineered to deliver recorded packets to the host with the greatest possible efficiency. That, together with the interstitial loss counter, is what makes the DAG so appealing for surveillance applications. The interstitial loss counter also finds application in forensics; a prosecutor needs to be able to prove that the record is complete or, if it is not, where it is not.