Interpreting analytics to improve content

Notes from the interpreting analytics breakout session.

Questions and answers

What can we do to clean up or prevent nonsensical feedback?

The way we approach it is to ignore or filter through the nonsensical feedback. We tend to focus on issues that are surfaced in multiple pieces of feedback. When we identify a trend, we determine the scope and impact. There currently is no concrete way to automatically filter nonsensical feedback.

Participant remark: I don't think tags are used effectively in the feedback modules. We need more ways to flag feedback that is nonsensical. A way to say “Don’t count this feedback.”

We have a page that gets lots of traffic but scores at 2.5 or lower. Although users seem to be finding the page, the page still scored low. Is there anything we can do about that?

There are a lot of things that factor into the page score. We do not calculate scores based on total traffic, but by percentage. If 500 people are going to that page that is great, but that is not the only indicator to look at when evaluating the page. If you have already tried to optimize and improve your content but are still scoring low, please feel free to send us a ServiceNow request so we can work with you to identify what may be the issues.

A lot of people click on ads, scan quickly, and if the information does not match what they are looking for they will leave. Scores are important, but they are only part of the story. Not often, but sometimes, there are extreme circumstances -- getting a lot of traffic yet generating a lot of negative feedback just because of the nature of the service.

We get a lot of responses through the feedback form at the bottom of our page. Unfortunately, constituents are not providing us with their email so it’s impossible to follow up on it. Is there anything we can do about this? People are sending us email about being pregnant and other personal information through the feedback form. We have tried to add our email to every page in hopes that they will use that to contact us.

There are a couple of modules you can use when building your page, there is one with the email field that you can use. There are options on the Org page called constituent communication option.

Participant remark: We have this page which is always scoring low regardless of how many other pages I've connected it to, or how much optimization or improvement I've taken towards the content. I am thinking about recreating this page as a new content type, from a Service Details page to an Information Details page.

After you engage with someone through the feedback form, does it improve the page? Often a lot of people are ending up on the wrong page and they are not finding what they want. I've had people reaching out to us looking for hairdresser for example, and it is obvious that they are on the wrong page.

In those cases, we have to assume that these are one-offs, otherwise we would start by looking at the funnel movement to understand where they are coming from.

Participant remark: Maybe if I saw the path they took to get there, maybe I’d see what I can do. The navigation on Mass.gov is massive and they could be clicking on anything.

If you go on Findability, you will find out which page they were on prior. We can do a bit more with Google Analytics, but unfortunately this is not as flexible.

When you are in the Analytics tab, does the information you are looking at makes sense?

Participant remark: It is a bit of a mystery, most of these are built to reflect measurable transactional outcomes. Whereas what we provide is more informational. When we are scored on a transactional basis, this does not truly reflect the page performance. So the score doesn't really make sense.

A couple of weeks ago, we released a new indicator for measuring conversion on informational pages. What this does is it takes a look at the engagement per page, scroll, clicks, and how much time is spent on the page.

Another approach to improve the content scoring would be to rethink the content type that will best serve the page, whether that is an Info Details or Service Details page. It may be worthwhile looking at a content type that will provide a better format for the information you are trying to present.

Participant remark: Needs improvement and traffic numbers, these are the pages that we look at most of the time in order to address any usability or content improvement issues.

Participant remark: On the news or media content type, I've had issues with the field for the media contact not showing up properly on this content type.

Participant remark: I've also had a similar issue when I add a media person and the system would sometimes not take it. Given this is a required field at times, it would hinder the work by preventing me from moving forward with the page configuration.

This is the first time I've heard of such an issue. I know I've said this a few times in the past but, whenever you are in the CMS and something is not working the way it should be, please feel free to file a ServiceNow request so that we can work on this with you. This not only helps us determine the magnitude of the bug and whether or not this is a one-off thing, but also helps surface issues that may be hindering authoring experience across the board.

Will the new Service page have more flexibility around what links need to go where?

Yes, absolutely. You can even include a file directly in the link, and most content types besides Org and Topic pages.

Is it best to clone when you are creating a new custom link group Service page?

Do that if you want to play around with the new format, in case you want to make any changes then push that to the actual live page. You would have to clone the page and request that one of them be changed to use the pilot template.

Is it possible to link halfway down the page using hashtags?

(No answer provided)

Is there a way to lock the pages or prevent 2 authors from working on the same page at the same time? In Percussion this is something that we used to be able to do and it proved really valuable because you can overwrite someone's work.

No, unfortunately. There are no restrictions built around that right now.

Looking at page scores, do you have any questions we can help answer?

Participant remark: Findability/connectivity, I don’t understand it. How does the score come about? Is it from a Service page to Service Detail or a Google search -- what does this score mean? Sometimes I make changes to the website and it goes up, sometimes it doesn't. Sometimes the score goes up without me doing anything. So I just don't do anything.

This was envisioned in part to help us better understand how effective our website architecture is. Findability helps us track if this architecture is working (Service page > Service Details). It’s not foolproof.

If you give someone a single page, and people access it directly, will it not get a high score? Whether it is through Google or Mass.gov or any other source?

Google counts. We track most entry points to any content on the website. If someone is coming from a PDF or document, that may not be captured. A long-term goal would be to identify what source you would like to track. If Service B is frequently receiving traffic from Service A, it might be an indicator that people are landing on Service A looking for Service B. It might be worthwhile to look for improvement within the language and wording of the content. Try to identify how we can help get people to where they want to go.

Participant remark: Findability is always our lowest score of all four. I try to improve this based on best practices and still fail. I've exhausted my options and the page is still scoring low, even as the content is optimized and linked to many other pages.

For instances like that please submit a ServiceNow ticket or come in for Office Hours, and we can analyze each page and try to identify what is going on.

Participant remark: It seems to be happening across the board though.

No's per 1,000 and broken links are the best way to determine your page score, whereas Findability is not always the best metric for measuring outcomes.

Participant remark: Outcomes are a problem for me.

We very recently released an outcome metric for information pages. These engagement metrics are designed for scenarios when users are coming to the page just to read some information compared to how it was previously measured as a transactional page.

Is it possible to measure outcomes for a page that is purely for personal vs professional use?

Not yet, but it is something we are hoping to implement soon. If one of the scores is significantly low, that can be an indicator that a better content type might be more effective. Each content type we designed with a specific purpose in mind. Outcomes is used to measure if each content type is performing as intended.