Finding Optimal K with the Silhouette Method in K-Means Clustering

Cluster analysis is an indispensable technique in exploratory data analysis (EDA). It can uncover unknown group structures inherent in data, such as customer segments, product groups, behavioral patterns, regional characteristics, and survey respondent types. However, clustering always involves a fundamental question: “How many clusters should be created?”

Especially in K-Means clustering, the number of clusters (K) must be determined before executing the algorithm. If K is too small, groups that are inherently different are merged together; if too large, a single group is unnaturally subdivided, making interpretation difficult. Representative methods to address this issue are the silhouette method and the elbow method. This article focuses on the silhouette method, detailing its principles, usage, interpretation precautions, and differences from the elbow method.

Basic Principles of the Silhouette Method

The silhouette method evaluates how well each data point fits into its assigned cluster. This method simultaneously assesses two questions:

• Are the points in the same cluster sufficiently close to each other?
• Are the points in different clusters sufficiently far from each other?

A good clustering result must satisfy that data within the same cluster are similar, while data in different clusters are distinctly different. The numerical representation of these conditions is the silhouette score.

Calculation and Interpretation of

Silhouette Scores

Silhouette scores range from -1 to 1. The specific calculation procedure is as follows.

First, for the data point of interest, compute the “cohesion” (a): the average distance to all other points within the same cluster. Next, compute the “separation” (b): the average distance to all points in the nearest neighboring cluster.

The silhouette score is defined as (b - a) / max(a, b). For example, if cohesion is 40 and separation is 80, the score is 0.5. The larger the separation and the smaller the cohesion, the larger the numerator becomes, and the score approaches 1. Conversely, if separation is small and cohesion is large, the score approaches 0 and may even become negative.

The interpretation of scores is as follows:

• Close to 1: Ideal state. The point fits well into its own cluster and is clearly separated from other clusters.
• Around 0: Ambiguous state. The point lies near the boundary between two clusters and could belong to either.
• Negative values: Possible misclassification. The point is closer to another cluster than to its assigned cluster.

Three Important Metrics for Cluster Evaluation

When applying the silhouette method, relying on a single metric is insufficient. In the Qiita article “How to Determine the Number of Clusters in K-Means? Finding the Optimal K with the Silhouette Method” (by Kanichiro Nishida, published June 14, 2026), it is recommended to combine the following three metrics for evaluation.

Average Silhouette Score

The average silhouette score across all observations. This is the most common indicator of overall clustering quality. For example, if the average is 0.42 for K=2, 0.51 for K=3, 0.47 for K=4, and 0.39 for K=5, then K=3 is a strong candidate. However, the average can hide some misclassifications or poorly separated clusters, so additional verification is needed.

Minimum Silhouette Score

The lowest silhouette score among all observations. It indicates “how bad the most problematic assignment is.” For example, if K=3 has an average of 0.42 and a minimum of -0.05, while K=4 has an average of 0.41 and a minimum of -0.48, then even though the averages are nearly the same, K=4 contains a very poor observation. Causes include outliers, noise, cluster overlap, or inappropriate K. However, because the minimum is heavily influenced by a single observation, it is appropriate to use it as a warning indicator.

Proportion of Negative Silhouette Scores

The proportion of data with silhouette scores below 0. It indicates “how many data points might be closer to another cluster.” If this proportion is high, the reliability of the clustering result requires caution.

Use Cases: Silhouette Method vs. Elbow Method

The elbow method plots the sum of squared distances (inertia) from each data point to the center of its assigned cluster while varying K, and takes the position where the decrease levels off (the “elbow”) as the optimal K. It is computationally light and intuitive, but difficult to judge when no clear elbow appears or when multiple elbows exist. In contrast, the silhouette method also considers separation between clusters, enabling more robust evaluation. In practice, it is advisable to use both methods together and cross-validate the results.

Silhouette Method Support in Exploratory

The same article also introduces silhouette method support for K-Means clustering recently added to the data analysis platform “Exploratory.” This allows users to explore the optimal K through GUI operations while viewing the three metrics (average, minimum, and negative scores) in a single list. As data analysis automation and visualization advance, leveraging such tools can greatly improve practical efficiency. For data visualization and sharing analysis results, the article How to Polish Mermaid Diagrams with Gemini Nano Banana is also useful.

Editorial Opinion

Short-term Impact

The silhouette method is a classic technique, but its integration into modern data analysis tools is progressing, making it more accessible to a broader range of practitioners. Over the next 3 to 6 months, it is expected that more cases will emerge where the silhouette method is provided as a standard feature on Exploratory and similar no-code/low-code analysis platforms. This will enable analysts who are not deeply familiar with machine learning to statistically justify the appropriate number of clusters. At the same time, there is a risk that erroneous practices of judging only by the average score could spread, highlighting the importance of education and outreach.

Long-term Perspective

Over a span of 1 to 3 years, we can expect automated optimization of clustering evaluation methods, including the silhouette method, to advance. In the context of AutoML, methods that solve K search as a silhouette score maximization problem may become standardized. Additionally, research on approximate computation and sampling-based fast silhouette methods for large-scale data is likely to progress, making practical calculations possible even for millions of data points. Furthermore, AI functions that visualize data points with low silhouette scores and explain the reasons in natural language to assist interpretation of clustering results are expected to be integrated.

Editorial Question

The silhouette method is not a panacea. In real business data, even when the theoretically optimal number of clusters is identified, there are many cases where a different K is chosen from the perspective of interpretability or operational cost. The editorial board asks readers: How should we strike a balance between statistical metrics and domain knowledge? In particular, if segments that are meaningful for business exist despite low silhouette scores, what criteria should be used to determine K? We hope for future discussions on best practices for applying the method to real data while understanding its limitations.

References

• How to Determine the Number of Clusters in K-Means? Finding the Optimal K with the Silhouette Method - Qiita — Published June 14, 2026
• Related: Exploratory official documentation (silhouette method support for K-Means clustering)