Last updated: July 20, 2021
Read time: 8 Minutes
Burp Sequencer employs standard statistical tests for randomness. These are based on the principle of testing a hypothesis against a sample of evidence, and calculating the probability of the observed data occurring, assuming that the hypothesis is true:
The significance level is a key parameter in this methodology. Using a lower significance level means that stronger evidence is required to reject the hypothesis that the tokens are randomly generated, and so increases the chance that non-random data will be treated as random. There is no universally "right" significance level to use for any particular purpose: scientific experiments often use significance levels in the region of 1% to 5%; the standard FIPS tests for randomness (which are implemented within Burp Sequencer) use significance levels in the region of 0.002% to 0.03%. Burp Sequencer lets you choose what significance level you wish to use to interpret its findings:
Some important caveats arise with any statistical-based test for randomness. The results may contain false negatives and positives for the following reasons:
Because of these caveats, the results of using Burp Sequencer should be interpreted only as an indicative guide to the randomness of the sampled data.
The character-level tests operate on each character position of the token in its raw form. First, the size of the character set at each position is counted - this is the number of different characters that appear at each position within the sample data. Then, the following tests are performed using this information:
Based on the above tests, the character-level analysis computes an overall score for each character position - this is the lowest probability calculated at each position by each of the character-level tests. The analysis then counts the number of bits of effective entropy for various significance levels. Based on the size of its character set, each position is assigned a number of bits of entropy (2 bits if there are 4 characters, 3 bits if there are 8 characters, etc.), and the total number of bits at or above each significance level are calculated.
The bit-level tests are more powerful than the character-level tests. To enable bit-level analysis, each token is converted into a set of bits, with the total number of bits determined by the size of the character set at each character position. If any positions employ a character set whose size is not a round power of two, the sample data at that position is translated into a character set whose size is the nearest smaller round power of two. The partial bit of data at the position is effectively merged into the whole bits derived from that position. This translation is done in a way that is designed to preserve the randomness characteristics of the original sample, without introducing or removing any bias. However, no process of this type can be perfect, and it is likely the process of analyzing samples with non-round character set sizes will introduce some inaccuracies into the analysis results.
When each token has been converted into a sequence of bits, the following tests are performed at each bit position:
Based on the above tests, the bit-level analysis computes an overall score for each bit position - this is the lowest probability calculated at each position by each of the bit-level tests. The analysis then counts the number of bits of effective entropy for various significance levels.