Why NPS Fluctuates Without Product Changes: Analyzing Non-Product Factors

When product managers and corporate executives use NPS (Net Promoter Score) as a key performance indicator, they often encounter perplexing questions like, “Why has the NPS dropped this month?” or “What are users dissatisfied with?” Development teams sometimes respond with, “We haven’t made any changes.” Many organizations break down metrics and analyze them thoroughly, from brand level to individual car models and specific features, yet still fail to identify the cause.

The reason behind this phenomenon lies in the fact that NPS does not solely measure product quality. The Publickey article “Why Has NPS Changed Despite No Product Changes?” (originally published on Huxiu.com, written by Daiwei Zhu) systematically organizes insights from the user experience industry to explain how non-product factors influence NPS. This article will explore the factors behind NPS fluctuations from a technical perspective, based on that analysis.

The Formula for User Experience

The basic structure of user experience can be expressed with the following formula:

User Experience = Product Capability - User Expectations

Even if the product’s performance remains unchanged, the user experience can fluctuate if user expectations change. These changes in expectations primarily occur through the following three channels:

Firstly, the emergence of competitive products. A prominent example in the automotive industry is when a new car model is launched with superior performance at a lower price compared to older models. Owners of the older model may feel “betrayed,” leading to a decline in their willingness to recommend. Even if the older car’s space, power, or in-car systems remain unchanged, its relative value diminishes, negatively impacting NPS.

Secondly, negative reputation or safety-related incidents can play a role. The deterioration of reputation in a specific product category can influence users’ expectations, even if the quality of individual products remains unaffected. For instance, media reports of accidents involving driver-assistance systems can heighten user anxiety, thus impacting NPS—even when the company’s own products are safe.

Thirdly, changes in trust towards the overall brand can affect NPS. Declines in customer support quality, privacy concerns, or corporate scandals can influence NPS independently of the evaluation of individual products.

Differences in Sample Composition

NPS surveys are based on sampling, and the methods used to select survey participants significantly influence the results. In practice, it’s often overlooked that the survey’s contact channel determines which user segments are covered.

Surveys conducted via SMS theoretically reach all owners, while surveys conducted through app push notifications tend to target active users who frequently engage with the app and have a high interest in the brand. These users are more likely to provide feedback and may focus more on aspects like smart features. Even when evaluating the same product, different survey channels lead to varying results.

Additionally, the duration of ownership is another crucial factor. Users who have owned a product for two months might provide feedback reflecting their initial excitement, while those who have owned it for a year base their evaluations on long-term experience. As ownership duration increases, the novelty tends to fade, and satisfaction often decreases. There is no singular correct data point; users at different lifecycle stages answer different questions. In survey design, it’s essential to consider which user group’s recommendations carry more weight.

Survivor Bias Skews Long-Term Data

In many product categories, NPS tends to increase once the ownership period exceeds three to five years. This phenomenon is attributed to survivor bias. Users dissatisfied with a product often switch early, leaving only loyal enthusiasts behind as respondents. Consequently, the survey sample is naturally filtered, and NPS artificially appears to rise.

This bias exists from the outset of surveys. NPS targets only existing customers, excluding those who decided against purchasing the product based on pre-purchase factors such as appearance, space, or usability. This explains why NPS scores are generally higher for dimensions like appearance and usability but lower for features like in-car systems or charging experiences, which require actual usage to evaluate. The appearance isn’t inherently better; users dissatisfied with it simply don’t buy the product.

This survivor bias is particularly critical in long-term tracking. Comparing NPS trends over time without accounting for changes in sample composition can lead to erroneous conclusions.

Reverse Causality Between Sales Volume and NPS

The conventional wisdom surrounding NPS suggests that a high score drives word-of-mouth promotion, leading to increased sales. However, in practice, reverse causality is also observed.

When sales volume decreases, hesitant buyers withdraw, leaving only passionate fans as purchasers. Consequently, NPS continues to rise. This apparent contradiction is common in product categories where opinions on aesthetics are divided. If sales decline while NPS remains high, conducting surveys among non-buyers can uncover the underlying reality.

The existence of this reverse causality highlights the danger of evaluating NPS as a standalone metric. The relationship between sales volume and NPS is bidirectional—one does not necessarily cause the other. Cross-analysis of both metrics enables a more accurate evaluation of the product.

Sample Size Reduction and Volatility

Declining sales volume presents another challenge. As NPS is based on sampling, smaller sample sizes lead to greater variability in results. For a product with 20,000 monthly sales, the sample pool is sufficiently large, yielding stable data. However, for a product with only 2,000 monthly sales, secondary and tertiary indicators may suffer from insufficient sample size, causing significant fluctuations based on the feedback of a single user.

This issue becomes particularly acute in analyses of highly specific functional dimensions. Evaluations of navigation accuracy, voice recognition quality, or Bluetooth connectivity stability can have even smaller sample sizes, increasing the risk of misinterpreting random noise as a genuine product issue.

Reinterpreting the NPS Metric: Practical Insights

This analysis demonstrates that NPS does not solely measure product quality but rather the synthesis of the following three elements:

• Product performance
• User expectations
• The market environment surrounding the product

Organizations using NPS as a KPI must systematically consider non-product factors rather than solely judging fluctuations based on product changes. Specifically, four aspects need to be verified:

Firstly, check for changes in the composition of survey participants. Track variables such as contact channels, survey timing, and user ownership durations.

Secondly, evaluate changes in the market environment. Consider trends in competing products, overall industry reputation, and fluctuations in corporate brand value.

Thirdly, validate the adequacy of the sample size. For analyses of specific dimensions, ensure that the sample size is statistically significant.

Fourthly, analyze the bidirectional relationship with sales volume. Cross-tabulate NPS and sales trends to exclude the possibility of reverse causality.

Editorial Opinion

A lack of awareness regarding non-product factors affecting NPS can lead to significant misjudgments in product management. In the short term, resource allocation may become inefficient due to incorrect problem identification, resulting in investments in unnecessary feature enhancements or misguided changes to sales strategies without proper market understanding. Product managers should redefine NPS as a “composite index of market receptiveness” rather than merely a “proxy for product quality.”

In the long term, over-reliance on NPS might hinder organizational learning. Incorrectly assuming that there are no issues with the product can deprioritize UX improvements, eroding competitive advantages over time. Annual long-term tracking must include cohort analyses and surveys of non-buyers to correct survivor bias. It is crucial for the industry to establish standard practices that incorporate control variables to explicitly account for non-product factors during NPS survey design.

References

• Why Has NPS Changed Despite No Product Changes? - Huxiu — Published on July 4, 2026
• Related article: Publickey “Why Has NPS Changed Despite No Product Changes?”