What does dividing by the square root of dimension normalize?

Reference inputs · synthetic data

Records
16
Data columns
4
Missing cells
0
Duplicate records
0
Feature duplicates
0

16 of 16 records · One record = one equally likely coordinate-product pattern. Statistics and downloads use all records.

About this dataset

Exact sign-product model: each coordinate product is independently −1 or +1 with equal probability before scaling the query. Enumerate all 2^d equally likely product patterns, without sampling. Keep the sign law fixed when changing dimension. Query and key coordinates originally have magnitude 1; multiplying only the query by c gives lengths c√d and √d. Variance uses the full distribution, with divisor 2^d.

  • This is an exact finite probability model, not a sampled training dataset. All 2^d sign-product patterns are equally likely. There are no missing products; repeated sums are intentional.
  • Independent fair signs for every original query and key coordinate induce these independent fair product signs. Many vector pairs have the same product pattern; each pattern represents the same number of pairs.
  • The raw dot product is c times the sum of the base products. Query and key norms are c√d and √d, so cosine divides the raw dot product by cd.
  • The explorer shows reference base products. Run reports include the products and settings for each run. Distribution variance divides by 2^d; generic explorer summaries describe the input columns and use sample standard deviation.

Counts exclude the row identifier. Duplicate counts are extra occurrences after the first. Feature duplicates ignore labels and experimental conditions. — means a statistic is undefined or not applicable.

Row
Stable record identifier, not a model feature.
Base product z1
Base product z1
Base product z2
Base product z2
Base product z3
Base product z3
Base product z4
Base product z4