Community Health Metrics are quantifiable measures that are used to track and assess the health of a Community in terms of how active it is, how it is evolving, and other inward-looking aspects of a Community’s lifecycle.
Community Health Metrics enable you to plan, manage, learn and feed back into the Community building process. They are also a key component for reports to senior management, stakeholders, and members.
When deciding what to measure, decide when to measure, which goals to measure against, which metrics/analytics match your objectives, and which Community platform metrics/analytics to use, whether using Yammer analytics, Swoop analytics, Adobe Analytics, or the analytics associated with your specific platform.
Community Health Metrics are quantifiable measures that are used to track and assess the health of a Community in terms of how active it is, how it is evolving, and other inward-looking aspects of a Community’s Lifecycle. It is important to distinguish these metrics from the Community Impact Indicators which are more outward-looking in that they describe what happens around a Community as a result of the Community’s operations.
Community Health Metrics
Community Impact Indicators
Inward-looking: they focus on the processes inside the community
Outward-looking: they focus on what happens around the community
They measure what a Community is, what it does, and how well it does it
They measure the what change is triggered in the environment around the Community as a result of the activities of the Community
They are very broad - as in: there can be many - and they are generally superficial
They focus narrowly on the desired impact of a Community - and they go deeper in trying to understand if, why, and how a specific impact happens
Attribution is generally easy: causes and consequences are easy to assess. E.g.: an increased number of members which takes place right after an onboarding campaign can safely be attributed to the success of the onboarding campaign.
Attribution is difficult. As Communities operate in a broad and complex environment, it is not easy to attribute a specific impact - e.g.: a changed behaviour, a new policy implemented - to the activities of a Community.
Examples: “number of members”; “number of participants to an event”; “number of posts on the Community’s forum”.
Example: “faster problem solving”; “improved peer review”; “increased awareness about X”.
Focus: operational. Monitoring these metrics is important as it is functional to continuously improving the health of a Community
Focus: strategic. Defining these indicators is important as it is functional to defining the identity and purpose of the Community. Monitoring and Reporting on these indicators can help securing support for the Community as it provides evidence that can be used to demonstrate the value of the Community
A metaphor: monitoring Community Performance Metrics is like monitoring a car’s gauges (such as: oil, fuel, water temperature): it tells you how the car is doing, but not where it is going. You may have a very healthy car and yet you may be going in the wrong direction.
A metaphor: monitoring Community Impact Indicators is like monitoring a car’s GPS: it tells you where the car is going, but not if it is functioning well. You may be going in the right direction, but not realize your car will break down before you get to your destination.
In terms of the Theory of Change, community internal metrics focus on activities/inputs and outputs.
Why are Community Health Metrics important?
Monitoring and analysing Community Health Metrics is important as it enables you to:
Plan – Defining success and setting a measurable goal is essential for planning your work and allocating scarce resources between competing projects, especially if you’re setting goals around growth in members, number of views, number of downloads, numbers of attendees at events, and so on.
Manage – It has often been said: “If you’re not measuring, you’re not managing.” Metrics are essential for assessing whether you are on the right track or whether you have to make adjustments in order to achieve your goals. You measure to help you know what is working well and what is not. It shows you what and where and when you need to make adjustments.
Learn – Metrics are essential for learning what went well, what did not go so well, what hit the mark, what missed the mark, and what unintended consequences emerged. Use metrics to learn from and contribute to a feedback loop to make the next iteration of the Community more successful.
Reports – It helps when creating Community Reports and sharing progress with community leaders, managers, stakeholders and sponsors which is key in maintaining and growing the support (and funding) for your Community. Regularly reporting to members also helps reassure and validate their participation by celebrating success along the way, and recognizing key contributors, both of which go to build a Sense of Community.
Recruitment – Successful measures also provide an incentive for others to join – everybody wants to join a successful Community.
How do you decide what to measure?
When deciding what to measure, you need to:
Decide when to measure.
Decide which goals to measure against.
Select appropriate metrics/analytics to match your objectives.
Select appropriate community platform metrics/analytics
Decide when to measure
In many processes, you will most frequently see the measurement and evaluation phase tacked on at the end after the implementation has taken place, for example, the long-established ADDIE training model (Analysis, Design, Development, Implementation, and Evaluation). Unfortunately, the evaluation stage is often given very little attention. Your design should always start with evaluation, the metrics you want to track and report
Decide which goals to measure against
Because you start your Community project by deciding on what to measure, you need to answer the following questions up front: How are you going to define success? What does success look like to you, to the beneficiaries, to your stakeholders? And how will you know that you’ve reached it? How will you be able to demonstrate the value of your efforts over time? You must decide on your goals before you can measure and decide on which metrics will best reflect that measurement of achievement of those goals.
You should take the following into account when choosing any metrics:
Shared value: The metrics must tie directly to your overall objectives, the value for the organization as well as addressing members’ needs, that is, the Shared Value.
Indicators: Measures should be simple to track and on a regular basis, (you need to decide how frequently you will measure. )
Baseline and benchmark: You need to establish a baseline in the process as soon as possible against which to measure your results (do not make the mistake of waiting until the end of a project or the end of a year to take your first measurement and then realize that you did not have a starting metric to see how far or not you have come.) You should track the same thing over time to determine progress.
Iterative: You should measure and refine and measure and refine regularly
Impact decision-making: Do not measure for the sake of measuring, but for the sake of changing / refining your Community building efforts (your platform will probably provide a plethora of statistics but use and report on only those that are meaningful)
Ownership: Identify who (from the Core Team) will be responsible for tracking.
Feed into reporting: Identify the audience for the data being collected. Decide how might it be used to influence senior managers, funders, and other stakeholders to help secure additional resources for the Community of Practice. See the article on Reporting for more information on this topic.
Theory of change: Make sure that your metrics are aligned with your Theory of Change.
Select the right metrics/analytics to match your objectives
Of course, you cannot look at the metrics revealed by analytics engines, or in fact any metrics, in isolation. Metrics should always be viewed in relation to your objectives. For example, if your objective is to reduce calls being made to your organization’s Technical Support group and have the Community answer those calls instead, then tracking the number of comments made on a blog will not have a lot of meaning. The metrics that you are interested in must relate to the objectives that you are trying to achieve. In the previous Technical Support example, you would be interested in metrics such as call deflections, response times, and quality of answers.
The following table shows examples of typical objectives and recommended types of metrics to track.
Type of Metric
To track…
Use these metrics:
Community vibrancy metrics
Overall health and utility of the Community
Membership growth, time-on-site, Community newsletter opt-ins and opt-outs
Engagement metrics
Community relevance
Logins and number of participating and contributing members, net new vs returning contributors, posts and replies per time interval, discussions and the depth of discussion
Content Consumption Impact Metrics
Content relevancy and reach
Content downloads, shares, and comments; even registrations, page views and the proportions of contributors vs consumers vs inactives
Customer Support Metrics
Value of Community as a support channel
Ticket deflection, time to resolution, number of “accepted” solutions
Organization Integration Metrics
Impact of Community on core operations
New ideas generated, new ideas implemented, increase in specific operational efficiencies such as faster implementation, or wider adoption of ideas
Tip: Make sure the metrics you use always relate to the objectives your Community is trying to achieve.
Select the right Community platform analytics/metrics
Most modern community software platforms include analytics tools to report on and help you assess how your Community is doing. There are several community platforms that you can choose from. This topic details the two main community platforms in use at the World Bank: the Yammer platform and the Collaboration 4 Development (C4D) platform, which is a custom version of Adobe Experience Manager (AEM) Community, and which hosts Communities4Dev.
Your organization may use different Online Platforms, in which case you will need to understand which metrics they support. Ultimately, your choice of metrics will be what metrics are supported by Online Platform that your Community runs on.
Find below a detailed breakdown of how metrics are handled on the WBG ecosystem of Online Platforms for Communities of Practice.
Disclaimer: this article is not meant as an endorsement of any of the platforms listed below.
Each platform has associated analytics engines producing metrics for activities on those platforms.
Community Platform
Analytics Tool
Yammer
Yammer Analytics
Swoop Analytics
C4D
Adobe Analytics
Yammer Analytics
Yammer produces several analytics reports, while Swoop sits on top of Yammer and provides a much richer set of data. Yammer provides a basic set of analytics that includes member activity, messages, and questions and answers:
People - Active people divided into active members, and active non-members categorized by whether they have posted messages, reacted to messages, and read messages.
Messages - Actual numbers and percentages of posted messages, read messages, and reacted to messages.
Questions and Answers – The number of questions and answers, the number of questions with the best answers, the answer ratio, (the ratio of unanswered to answered questions), and the rate of new questions over a specified period and the corresponding rate of answers.
Swoop Analytics
The Swoop application, which sits on top of Yammer, provides a much richer set of data at three different levels: the individual, the Community, and the enterprise.
Swoop Individual Personal Metrics
Individual Swoop metrics provide individuals with an idea of how they compare with other individuals in the Community, how their posts are received by the Community, shows impact on the network and how influential the individual is in the Community.
Swoop Community, Group, and Team Metrics
Community, Group, and Team Swoop metrics enable you to assess how your group is collaborating. They let you set goals, track the progress of your Community achieving those goals, and help you drive results.
C4D/Adobe Analytics
The Collaboration 4 Development (C4D) platform is a customized version of Adobe Experience Manager (AEM) Community platform. You use Adobe Analytics to examine activities on C4D. Adobe Analytics is a very robust analytics engine and has over 300 individual measures of community activity.
Some of the more commonly used metrics include:
Visitors, unique visitors and return visitors, and where they have come from.
Page views and visits
Time on site, time spent per visit, and time spent per visitor
Members - numbers and increase/decline in membership rates over time
Blog » Community Health Metrics
Community Health Metrics
Summary:
Resources:
Template: Community M&E Framework
What are Community Health Metrics?
Community Health Metrics are quantifiable measures that are used to track and assess the health of a Community in terms of how active it is, how it is evolving, and other inward-looking aspects of a Community’s Lifecycle. It is important to distinguish these metrics from the Community Impact Indicators which are more outward-looking in that they describe what happens around a Community as a result of the Community’s operations.
Community Health Metrics
Community Impact Indicators
Inward-looking: they focus on the processes inside the community
Outward-looking: they focus on what happens around the community
They measure what a Community is, what it does, and how well it does it
They measure the what change is triggered in the environment around the Community as a result of the activities of the Community
They are very broad - as in: there can be many - and they are generally superficial
They focus narrowly on the desired impact of a Community - and they go deeper in trying to understand if, why, and how a specific impact happens
Attribution is generally easy: causes and consequences are easy to assess. E.g.: an increased number of members which takes place right after an onboarding campaign can safely be attributed to the success of the onboarding campaign.
Attribution is difficult. As Communities operate in a broad and complex environment, it is not easy to attribute a specific impact - e.g.: a changed behaviour, a new policy implemented - to the activities of a Community.
Examples: “number of members”; “number of participants to an event”; “number of posts on the Community’s forum”.
Example: “faster problem solving”; “improved peer review”; “increased awareness about X”.
Focus: operational. Monitoring these metrics is important as it is functional to continuously improving the health of a Community
Focus: strategic. Defining these indicators is important as it is functional to defining the identity and purpose of the Community. Monitoring and Reporting on these indicators can help securing support for the Community as it provides evidence that can be used to demonstrate the value of the Community
A metaphor: monitoring Community Performance Metrics is like monitoring a car’s gauges (such as: oil, fuel, water temperature): it tells you how the car is doing, but not where it is going. You may have a very healthy car and yet you may be going in the wrong direction.
A metaphor: monitoring Community Impact Indicators is like monitoring a car’s GPS: it tells you where the car is going, but not if it is functioning well. You may be going in the right direction, but not realize your car will break down before you get to your destination.
In terms of the Theory of Change, community internal metrics focus on activities/inputs and outputs.
Why are Community Health Metrics important?
Monitoring and analysing Community Health Metrics is important as it enables you to:
Plan – Defining success and setting a measurable goal is essential for planning your work and allocating scarce resources between competing projects, especially if you’re setting goals around growth in members, number of views, number of downloads, numbers of attendees at events, and so on.
Manage – It has often been said: “If you’re not measuring, you’re not managing.” Metrics are essential for assessing whether you are on the right track or whether you have to make adjustments in order to achieve your goals. You measure to help you know what is working well and what is not. It shows you what and where and when you need to make adjustments.
Learn – Metrics are essential for learning what went well, what did not go so well, what hit the mark, what missed the mark, and what unintended consequences emerged. Use metrics to learn from and contribute to a feedback loop to make the next iteration of the Community more successful.
Reports – It helps when creating Community Reports and sharing progress with community leaders, managers, stakeholders and sponsors which is key in maintaining and growing the support (and funding) for your Community. Regularly reporting to members also helps reassure and validate their participation by celebrating success along the way, and recognizing key contributors, both of which go to build a Sense of Community.
Recruitment – Successful measures also provide an incentive for others to join – everybody wants to join a successful Community.
How do you decide what to measure?
When deciding what to measure, you need to:
Decide when to measure
In many processes, you will most frequently see the measurement and evaluation phase tacked on at the end after the implementation has taken place, for example, the long-established ADDIE training model (Analysis, Design, Development, Implementation, and Evaluation). Unfortunately, the evaluation stage is often given very little attention. Your design should always start with evaluation, the metrics you want to track and report
Decide which goals to measure against
Because you start your Community project by deciding on what to measure, you need to answer the following questions up front: How are you going to define success? What does success look like to you, to the beneficiaries, to your stakeholders? And how will you know that you’ve reached it? How will you be able to demonstrate the value of your efforts over time? You must decide on your goals before you can measure and decide on which metrics will best reflect that measurement of achievement of those goals.
You should take the following into account when choosing any metrics:
Shared value: The metrics must tie directly to your overall objectives, the value for the organization as well as addressing members’ needs, that is, the Shared Value.
Indicators: Measures should be simple to track and on a regular basis, (you need to decide how frequently you will measure. )
Baseline and benchmark: You need to establish a baseline in the process as soon as possible against which to measure your results (do not make the mistake of waiting until the end of a project or the end of a year to take your first measurement and then realize that you did not have a starting metric to see how far or not you have come.) You should track the same thing over time to determine progress.
Iterative: You should measure and refine and measure and refine regularly
Impact decision-making: Do not measure for the sake of measuring, but for the sake of changing / refining your Community building efforts (your platform will probably provide a plethora of statistics but use and report on only those that are meaningful)
Ownership: Identify who (from the Core Team) will be responsible for tracking.
Feed into reporting: Identify the audience for the data being collected. Decide how might it be used to influence senior managers, funders, and other stakeholders to help secure additional resources for the Community of Practice. See the article on Reporting for more information on this topic.
Theory of change: Make sure that your metrics are aligned with your Theory of Change.
Select the right metrics/analytics to match your objectives
Of course, you cannot look at the metrics revealed by analytics engines, or in fact any metrics, in isolation. Metrics should always be viewed in relation to your objectives. For example, if your objective is to reduce calls being made to your organization’s Technical Support group and have the Community answer those calls instead, then tracking the number of comments made on a blog will not have a lot of meaning. The metrics that you are interested in must relate to the objectives that you are trying to achieve. In the previous Technical Support example, you would be interested in metrics such as call deflections, response times, and quality of answers.
The following table shows examples of typical objectives and recommended types of metrics to track.
Type of Metric
To track…
Use these metrics:
Community vibrancy metrics
Overall health and utility of the Community
Membership growth, time-on-site, Community newsletter opt-ins and opt-outs
Engagement metrics
Community relevance
Logins and number of participating and contributing members, net new vs returning contributors, posts and replies per time interval, discussions and the depth of discussion
Content Consumption Impact Metrics
Content relevancy and reach
Content downloads, shares, and comments; even registrations, page views and the proportions of contributors vs consumers vs inactives
Customer Support Metrics
Value of Community as a support channel
Ticket deflection, time to resolution, number of “accepted” solutions
Organization Integration Metrics
Impact of Community on core operations
New ideas generated, new ideas implemented, increase in specific operational efficiencies such as faster implementation, or wider adoption of ideas
Table based on data in the Higher Logic (platform provider) document “How to Build a Thriving Online Community.”
Tip: Make sure the metrics you use always relate to the objectives your Community is trying to achieve.
Select the right Community platform analytics/metrics
Most modern community software platforms include analytics tools to report on and help you assess how your Community is doing. There are several community platforms that you can choose from. This topic details the two main community platforms in use at the World Bank: the Yammer platform and the Collaboration 4 Development (C4D) platform, which is a custom version of Adobe Experience Manager (AEM) Community, and which hosts Communities4Dev.
Your organization may use different Online Platforms, in which case you will need to understand which metrics they support. Ultimately, your choice of metrics will be what metrics are supported by Online Platform that your Community runs on.
Find below a detailed breakdown of how metrics are handled on the WBG ecosystem of Online Platforms for Communities of Practice.
Disclaimer: this article is not meant as an endorsement of any of the platforms listed below.
Each platform has associated analytics engines producing metrics for activities on those platforms.
Community Platform
Analytics Tool
Yammer
Yammer Analytics
Swoop Analytics
C4D
Adobe Analytics
Yammer Analytics
Yammer produces several analytics reports, while Swoop sits on top of Yammer and provides a much richer set of data. Yammer provides a basic set of analytics that includes member activity, messages, and questions and answers:
People - Active people divided into active members, and active non-members categorized by whether they have posted messages, reacted to messages, and read messages.
Messages - Actual numbers and percentages of posted messages, read messages, and reacted to messages.
Questions and Answers – The number of questions and answers, the number of questions with the best answers, the answer ratio, (the ratio of unanswered to answered questions), and the rate of new questions over a specified period and the corresponding rate of answers.
Swoop Analytics
The Swoop application, which sits on top of Yammer, provides a much richer set of data at three different levels: the individual, the Community, and the enterprise.
Swoop Individual Personal Metrics
Individual Swoop metrics provide individuals with an idea of how they compare with other individuals in the Community, how their posts are received by the Community, shows impact on the network and how influential the individual is in the Community.
Swoop Community, Group, and Team Metrics
Community, Group, and Team Swoop metrics enable you to assess how your group is collaborating. They let you set goals, track the progress of your Community achieving those goals, and help you drive results.
C4D/Adobe Analytics
The Collaboration 4 Development (C4D) platform is a customized version of Adobe Experience Manager (AEM) Community platform. You use Adobe Analytics to examine activities on C4D. Adobe Analytics is a very robust analytics engine and has over 300 individual measures of community activity.
Some of the more commonly used metrics include:
This article is part of the WBG Communities of Practice Toolkit licensed under a Creative Commons Attribution 4.0 International License. The toolkit features practical resources to help you develop impactful Communities of Practice. 📖 Learn more about the Toolkit. ▶ Access the Toolkit.