Multi-attribute decision-making (MADM) methods are essential tools for evaluating, ranking, and selecting options based on various criteria. Surprisingly, many of us use these techniques intuitively, without even realizing the complex math behind them. Imagine you have three alternatives and three criteria to consider. The simplest and most popular approach is to rate each criterion for every alternative.
| Criterion 1 | Criterion 2 | Criterion 3 | |
| Alternative 1 | Fast | A | 1 |
| Alternative 2 | Slow | B | 3 |
| Alternative 3 | Fast | A | 1 |
These alternatives are tricky to compare since they mix nominal (A, B) and categorical (Fast, Slow) variables. Categorical variables follow a specific order. Enter encoding procedures: they transform these variables into numerical values, making comparison a breeze.
| Criterion 1 | Criterion 2 | Criterion 3 | |
| Alternative 1 | 2 | 1 | 1 |
| Alternative 2 | 1 | 2 | 3 |
| Alternative 3 | 2 | 1 | 1 |
Next, we need to decide if we’re dealing with cost or benefit attributes. For benefits, more is always better. Think of it this way: “Fast” is more desirable, while “A” is less favored compared to “B.” Beyond simple nominal and categorical variables, we have complex ones like cyclical features (e.g., months of the year, hours of the day), which we encode using sine and cosine transformations to capture their repeating patterns.
To compare values effectively, normalization is key. Without it, criteria with larger ranges can distort the results. A popular method for this is linear scale transformation:
Next, a weight is introduced for each criterion.
| Criterion 1 | Weight 1 | Criterion 2 | Weight 2 | Criterion 3 | Weight 3 | |
| Alternative 1 | 2 | 1 | 1 | 2 | 1 | 3 |
| Alternative 2 | 1 | 1 | 2 | 2 | 3 | 3 |
| Alternative 3 | 2 | 1 | 1 | 2 | 1 | 3 |
To avoid imbalance, a probability distribution (0 to 1) with a sum of 1 is used for all criteria weights.
| Criterion 1 | Weight 1 | Criterion 2 | Weight 2 | Criterion 3 | Weight 3 | |
| Alternative 1 | 2 | 0.1 | 1 | 0.3 | 1 | 0.6 |
| Alternative 2 | 1 | 0.1 | 2 | 0.3 | 3 | 0.6 |
| Alternative 3 | 2 | 0.1 | 1 | 0.3 | 1 | 0.6 |
Summarizing the weighted values for each alternative helps in making a selection. For example, for Alternative 1:
In transformer networks, this resembles the additive (Bahdanau) attention mechanism, where attention scores are multiplied by attention weights. In comparison, the multiplicative (dot-product) attention mechanism introduced by Luong et al. calculates:
Many other MADM approaches are feasible. For example, consider prediction in language translation, where criteria are hidden states of already seen words and alternatives are words in the vocabulary.
| Word no. 1 of input sequence | Attention Score x Softmax | |
| Word 1 of target vacubular | 2 | 1 |
| Word 2 of target vacubular | 1 | 1 |
| Word 3 of target vacubular | 2 | 1 |
The result is the so-called context vector. Unlike former RNN sequence-to-sequence approaches, the attention mechanism considers all already seen input words, also known as hidden states.