In CRO, many decisions are based on data: conversion rates, funnels, clicks, form submissions or revenue generated. All of these data points are necessary to understand how a website is performing, but they do not always explain what is happening on their own. We may know that a page converts worse on mobile or that many users abandon the checkout, but that does not automatically mean we know what the problem is.
That is why, when we talk about conversion rate optimisation, it is important to combine quantitative analysis with qualitative analysis. The first helps us detect patterns, measure behaviour and identify points of friction. The second allows us to get closer to the user’s context and better understand their doubts, blockers or motivations. Neither approach works in isolation in every case, but together they allow us to make better-informed decisions.
Contents
What is quantitative analysis?
Quantitative analysis in CRO consists of reviewing numerical data to understand how users behave on a website. Instead of starting from an opinion or a feeling, this type of analysis relies on metrics such as conversion rate, traffic, clicks, form abandonment, completed funnel steps or revenue generated.
This type of analysis helps answer questions such as which pages are performing worse, at which step of the process more users drop off, or whether there are relevant differences between devices, channels or user types. For example, if an online store detects that mobile conversion is much lower than desktop conversion, quantitative analysis makes it possible to identify that difference and measure its impact.
However, this data does not always explain the exact reason behind the problem. It may show that many users abandon a specific step of the checkout, but it does not necessarily tell us whether they do so because the form is too long, because shipping costs appear too late or because there is an element that creates distrust. For that reason, quantitative analysis is usually a good starting point to detect optimisation opportunities, but it is not always enough to fully understand them.
What is qualitative analysis?
Qualitative analysis in CRO focuses on better understanding the context, doubts and difficulties that users may have when browsing a website.
Unlike quantitative analysis, it is not based so much on aggregated metrics, but on signals that are closer to the user’s behaviour or opinion. For example, session recordings, surveys, interviews, user tests or comments received through forms and feedback tools. This type of analysis helps answer questions that numerical data cannot always explain on its own.
We may know that a page has a low conversion rate, but perhaps we need to see how users interact with that page to understand what may be happening. Maybe they do not see an important button, they do not understand an explanation, they have doubts about the price or they run into an unclear error message. That is why qualitative analysis is usually very useful when we have already detected a problem, but still do not know what the cause might be.
It will not always give us a definitive answer, and it is also not a good idea to draw general conclusions from one or two isolated cases. However, it can help us find patterns, discover frictions we had not considered and formulate more specific hypotheses to improve the website.
Difference between quantitative and qualitative analysis
The main difference between quantitative and qualitative analysis lies in the type of question they help us answer.
Quantitative analysis helps us understand what is happening. For example, which pages have lower conversion, what percentage of users abandon a form or on which device there is a clearer drop.
Qualitative analysis, on the other hand, helps us understand why it may be happening. It will not always give us a definitive explanation, but it can bring us closer to the user’s context.
For example, an analytics tool may show that many users leave a product page without adding anything to the cart. That data is useful, but it still leaves many questions open. The price may not be clear, information about shipping costs may be missing, the images may not be enough or the add-to-cart button may go unnoticed.
This is where qualitative analysis can provide more context. Reviewing session recordings, launching a survey or analysing user comments can help us better understand what friction is behind that behaviour. That is why both approaches should not be seen as alternatives. In a CRO process, they usually complement each other. Quantitative analysis helps detect and prioritise problems. Qualitative analysis helps interpret them and turn them into more specific hypotheses.
Quantitative data sources in CRO
Quantitative data sources are those that allow us to measure user behaviour on a website. Usually, this data comes from digital analytics tools such as Google Analytics, Adobe Analytics, Matomo or Piwik PRO. It can also come from internal tools, e-commerce platforms, CRMs or proprietary databases.
What matters is not so much the specific tool, but the type of information we can analyse with it. In CRO, this data is usually used to review pages, funnels, forms, events, revenue, products, campaigns or user segments.
For example, we can analyse whether a page receives a lot of traffic but converts poorly, whether a form has a significant drop-off on a specific field or whether performance changes a lot between mobile and desktop.
It is also common to work with conversion funnels. A funnel allows us to represent the steps we expect a user to follow until they complete an action. In an online store, for example, we could review how many users view a product, how many add it to the cart, how many start the checkout and how many end up buying.
This type of analysis helps locate points of friction more clearly. If the biggest drop happens between the cart and the start of the checkout, it probably makes more sense to investigate that part before spending time on pages that do not seem to have such an obvious problem.
In addition, quantitative data allows us to segment the analysis. It is not always a good idea to look only at the overall average of the website. Sometimes, a problem only appears on a specific device, browser, country, acquisition channel or user type. If we do not review those segments, we may miss important problems or misinterpret the data.
That is why quantitative sources are especially useful for detecting where there may be an optimisation opportunity and how much impact it could have. They do not always tell us what the cause is, but they do help us better prioritise where it is worth investigating.
Qualitative data sources in CRO
Qualitative data sources are those that help us better understand how users experience a specific journey on a website. They do not focus so much on measuring how many people do something, but on observing what they do, what they say or what difficulties they seem to have while browsing.
Within this type of analysis, we can find tools such as session recordings, heatmaps, on-site surveys, user tests, interviews or comments received through forms, chats or customer support teams.
For example, a session recording may show that a user tries to click several times on an element that is not clickable. That behaviour may not appear clearly in an analytics tool, but it may indicate that the design is creating the wrong expectation.
Surveys can also be useful when we want to understand specific user doubts. If someone leaves a product page, we will not always know the reason by only reviewing conversion data. However, a simple question such as “is there anything stopping you from completing the purchase?” can reveal problems related to price, shipping costs, delivery times or lack of information.
It is also worth taking into account the information that already exists inside the company. Support, sales or customer service teams often receive questions, objections and complaints that can be very useful for CRO. Sometimes, those conversations reveal problems that have not yet become a clear data point inside an analytics tool.
These sources should not be interpreted as an absolute truth. A recording or a survey response can be very useful, but it does not always represent all users. For that reason, the goal should not be to draw general conclusions from an isolated case, but to detect patterns, understand possible frictions and build hypotheses that we can then validate more effectively.
Limitations of quantitative analysis
Quantitative analysis is very useful for detecting problems, but it has an important limitation: it does not always explain why they happen.
An analytics tool may show that a page has a low conversion rate, that a form loses users at a specific step or that mobile performance is worse than desktop. However, that data alone does not usually explain the exact cause.
For example, if many users abandon a checkout at the payment step, we can know that there is a problem there. But we will not automatically know whether the reason is related to the payment methods available, lack of trust, a technical error, added costs or any other friction.
It can also happen that the data leads us to an interpretation too quickly. If a page has few clicks on a button, we might think that users are not interested in the CTA. But perhaps the problem is that the button is not visible enough, that it appears too far down the page, that the text is not clear or that the user still does not have enough information to move forward.
Another common risk is analysing only aggregated data. The overall average of a website can hide important problems in specific segments. An apparently stable conversion rate may hide a drop on mobile, in a specific browser, in a particular country or among users coming from a specific campaign.
That is why quantitative analysis should not be used as a final answer, but as a way to guide the research. It helps us know where to look, what to prioritise and what impact a problem may have. But in many cases, we will need to complement it with qualitative analysis to better understand what is happening behind those numbers.
Limitations of qualitative analysis
Qualitative analysis provides a lot of context, but it also has limitations. The most important one is that it usually works with small samples. We can review several session recordings, read survey responses or run user tests, but that does not mean that all users on the website behave in the same way.
For example, a recording may show one person having trouble finding a button. That case can be very interesting, especially if it helps us detect a friction we had not seen before. But we should not automatically assume that all users have the same problem.
There is also the risk of focusing on examples that confirm a previous idea. If we already believe that a page has a specific problem, it is easy to pay more attention to the recordings, responses or comments that fit that interpretation. That is why qualitative analysis requires some caution and should not be used only to confirm what we already thought.
Surveys can also lead to unreliable conclusions if the questions are not well designed. A question that is too leading can influence the user’s answer. And a survey that is too long may mean that only the most motivated people respond, usually because they have had an especially good or especially bad experience.
That is why qualitative analysis works best when it is used to detect patterns and generate hypotheses, not to measure the exact impact of a problem. It can help us better understand what may be happening, but we will usually need to compare it with quantitative data or with an experiment if we want to know how many users are affected or what real impact an improvement may have.
How to combine quantitative and qualitative analysis in CRO
In a CRO process, quantitative and qualitative analysis should not be used separately. Usually, one helps complete what the other cannot explain on its own. Quantitative data helps us detect where there may be a problem, while qualitative data helps us better understand what may be happening at that point.
For example, an analytics tool may show that many users leave a product page without adding anything to the cart. That data already tells us that there is something worth reviewing. However, we still need to better understand what may be holding the user back.
At that point, we could review session recordings, launch a survey on the page or analyse comments received through other channels. Maybe we discover that users cannot find delivery information, that the images do not answer their questions or that the main button does not stand out enough.
From there, the next step should not be to apply changes without further thought. Ideally, that learning should be turned into a specific hypothesis. For example, if we believe that users are not moving forward because they have doubts about delivery times, we could propose that showing that information more clearly near the purchase button will reduce friction and increase add-to-cart actions.
After that, the hypothesis can be prioritised and validated in the most appropriate way. In some cases, it will make sense to launch an A/B test. In others, it may be enough to apply the change and monitor its impact, especially if traffic is low or if the problem detected is clearly a usability friction.
What matters is not treating analysis as an isolated step. The goal is not only to find interesting data, but to turn that data into better-informed decisions. First we detect the problem, then we try to understand it better, we formulate a hypothesis and, finally, we measure whether the proposed solution actually improves the user experience or the website’s performance.
When to use each type of analysis
It is not always necessary to start the analysis in the same way. In some cases, it makes more sense to begin with quantitative data. In others, it may be more useful to first review qualitative information to better understand what doubts or frictions users are experiencing.
Quantitative analysis is usually especially useful when we want to measure the size of a problem. For example, if we want to know which step of a funnel has the highest abandonment, whether conversion has dropped on a specific device or whether a page performs worse than other similar pages, we need numerical data.
It is also the right approach when we want to prioritise. If we have several pages or processes that could be improved, quantitative data helps us decide where there may be a more relevant opportunity. Optimising a page with a lot of traffic and low conversion is not the same as dedicating the same effort to a page with very few sessions.
Qualitative analysis, on the other hand, is usually more useful when we need to better understand the user’s context. If we already know that a problem exists, but we are not sure why it is happening, reviewing recordings, surveys, user tests or comments can help us find possible explanations.
It can also be a good starting point when we still do not know what to look for. For example, if we are analysing a new page, a redesign or a process with little historical data, qualitative analysis can help us detect initial frictions before there is enough quantitative data to draw conclusions.
In practice, the most common approach is to use both methods at different moments of the process. We can start with quantitative data to detect an opportunity, use qualitative analysis to understand it better and then return to quantitative data to measure whether the solution applied has worked.
What matters is not choosing between one type of analysis or the other, but understanding which question we want to answer at each moment. If we want to know how often something happens, we will need quantitative data. If we want to better understand why it may be happening, we will need qualitative information.
In summary, quantitative and qualitative analysis have different roles within a CRO process. The first helps us detect what is happening and measure the size of a problem. The second allows us to better understand the user’s context and the possible reasons behind that behaviour.
That is why the most useful approach is usually not to choose between one or the other, but to combine them. The more data we have to understand a problem from different perspectives, the easier it will be to formulate realistic hypotheses and make optimisation decisions with a bit more judgement.
