Understanding Scalarization Functions for Multi-Objective Optimization
Written on
Chapter 1: Introduction to Scalarization Functions
Scalarization functions are pivotal in the field of optimization and decision-making, acting as a conduit between intricate, multi-dimensional challenges and simpler, solvable forms. Although the concept may appear daunting, it is likely that you have encountered it before.
Scalarization functions convert vectors into scalar values, with scalar values being numerical representations of magnitude, such as temperature, mass, and length. This conversion is where the term "scalarization function" originates, as these methods scalarize vectors into single values.
Unlike scalars, vectors possess both magnitude and direction; for instance, velocity is characterized by speed and direction. Another term for scalarization functions is decomposition functions.
To illustrate, consider the weighted sum, which serves as a straightforward example of a scalarization function that many may have encountered during their academic years. While it may evoke stressful memories of exams, this formula has practical applications in calculating final grades. Here's a quick breakdown:
Suppose three assessments contribute to the final grade:
- Assignments: 30% weight
- Midterm exams: 40% weight
- Final exam: 30% weight
If your scores are:
- Assignments: 95/100
- Midterm exams: 85/100
- Final exam: 90/100
You can compute the final grade using the weighted sum formula:
Weighted Average = (Weight of Assignments * Score in Assignments) +
(Weight of Midterm Exams * Score in Midterm Exams) +
(Weight of Final Exam * Score in Final Exam)
Calculating this gives:
Weighted Average = (0.30 * 95) + (0.40 * 85) + (0.30 * 90)
= 28.5 + 34 + 27
= 89.5
In this calculation, we have transformed three individual scores into a single aggregated score, thus reducing the dimensionality of the assessment. This provides a clear metric for evaluating a student's performance.
Previously, students faced three objectives to maximize. However, the weighted sum method distills this into a single target to optimize, which simplifies their academic goals. Additionally, without this scalarization process, determining whether a student meets the required standards would be challenging; the weighted sum allows for efficient assessment.
Before diving deeper, let’s visualize scalarization functions using a two-exam scenario.
The first video titled "Introduction to Scalarization Methods for Multi-objective Optimization" provides foundational insights into these concepts, showcasing their utility in optimization problems.
The visual representation in Figure 1 showcases weights of [0.5, 0.5], illustrating equal importance for both exams, while Figure 2 shifts to weights of [0.8, 0.2], indicating a heavier emphasis on Exam 1. The gradient intensity here exemplifies how changes in weights can significantly impact the final weighted sum.
This visual aids in understanding the minimum and maximum points. The minimum point represents the lowest score possible, while the maximum corresponds to the ideal scenario. Although these points may not be essential for basic computations, they become crucial in more complex scalarization functions.
At a high level, we define a scalarization function (SF) as follows:
SF(f, w) = scalar_value
Adjusting either the weights or the scores alters the final output, emphasizing the need for consistency in weightings when calculating grades. This ensures fairness across assessments.
In this article, I will explore some lesser-known scalarization functions, their purposes, and provide Python implementations.
Chapter 2: Applications of Scalarization Functions
Multi-Objective Optimization
Scalarization functions are instrumental in multi-objective optimization, transforming multiple scores into a single objective problem. We previously discussed how students, instead of maximizing their individual assessments, focus on maximizing their weighted average.
By converting multi-objective challenges into single-objective problems, scalarization functions facilitate the application of various optimization techniques, allowing for the identification of Pareto optimal solutions.
The second video titled "Part 1-Road To Learn Finetuning LLM With Custom Data-Quantization, LoRA, QLoRA Indepth Intuition" delves deeper into the intricacies of optimization techniques.
Understanding Pareto optimality is essential in this context. For example, if you allocate your study time between two subjects—maths and English—achieving a Pareto optimal outcome means maximizing scores in one subject without sacrificing the other. The ability to plot these scores helps visualize potential trade-offs.
In summary, scalarization functions are vital tools in multi-objective optimization, streamlining complex decision-making processes across various disciplines.