CivicPlus Agent Analytics provides insight into agent performance, identifies knowledge and content gaps, and surfaces resident engagement trends.
Who can use this feature?
Organization Owners
Instructions
Select AI Agents in the navigation menu and click Agent Analytics
Select the agent you want to view analytics for in the Filter by menu
View analytics for the agent
Dashboard Overview: High-level conversation outcome metrics are displayed on page load, including total conversations, unique users, and outcome breakdowns. Data represents a rolling 30-day window and is updated nightly.
Trending Topics: Identify what residents are asking about most frequently. Click any topic to open a modal view of related conversations. Learn more about trending topics.
Content Gaps: Surface areas where the agent lacks sufficient knowledge to answer resident questions. Click any gap to view the related conversations, with horizontal scrolling for conversations and vertical scrolling for messages. Learn more about content gaps.
Knowledge Sources & Citations: View which content sources the agent cites in responses, bucketed by type. Click a source type to filter citations from most to least cited. Each citation link opens in a new browser tab.
Content Gaps
A Content Gap is a topic where at least one conversation ended without meeting the resident's need. Content Gaps are shown grouped by topic (such as "Parking," "Trash & Recycling").
How does a conversation end up contributing to a Content Gap?
After a conversation ends (60 minutes of inactivity), the system reviews the full transcript and asks one question: Did the resident ultimately get what they needed?
This is an AI judgment call, not a simple rule like "no links = content gap." The system reads the entire conversation and decides whether the resident's need was met by the end.
What outcomes are possible?
Outcome | What it Means | Counts as a Gap? |
|---|---|---|
Resolved | The resident got a useful, specific answer. | No |
Dropped Off | The agent responded appropriately (asked a clarifying question), but the resident stopped replying. | No |
Unresolved | The resident's need was not met, such as vague answers, unhelpful redirects, or repeated deflection without specifics. | Yes |
Guardrail | The request was off-topic, abusive, or out of scope and was correctly blocked. | No |
Spam | The input was gibberish, a test, or not a real question. | No |
Why might a conversation be flagged as "Unresolved" even if the agent provided citations?
The system looks at the overall outcome, not just whether links were provided. A conversation can still be marked unresolved if:
The agent provided citations but never directly answered the question.
The agent gave a vague response and ended with an open-ended offer to help further, without confirming the resident's need was met.
The resident expressed continued confusion or dissatisfaction.
In short: Citations alone don't prevent a content gap. What matters is whether the conversation ended with the resident's question clearly answered.
Why does this sometimes feel too broad?
The current logic errs on the side of flagging more conversations rather than fewer. For example:
If the agent provides several relevant citations and then says "Is there anything else I can help you with?" but the resident never replies, the system may still classify this as unresolved rather than dropped off, depending on whether the transcript shows the original question was directly answered.
We're aware this can surface conversations that feel "fine" to anyone reviewing them. This is by design for now. We'd rather flag too many and tune based on your feedback than miss real gaps.
How are topics determined?
A separate step assigns each conversation to a topic bucket (such as Permits & Inspections, Water & Utilities). The Content Gaps view groups unresolved conversations by these topics so you can see which areas need attention.
What can I do with this information?
Right now, Content Gaps help you identify where your agent struggles most so you can:
Add or improve knowledge sources for those topics.
Identify patterns (such as residents asking questions your content doesn't cover).
Provide feedback to us on conversations that don't seem like real gaps, this helps us tune the logic.
Trending Topics
Trending Topics shows you which subject areas residents are asking about most frequently. Topics are displayed as a ranked list in which the top list item represents the most-asked-about topic. The number of conversations within each topic is displayed per row.
How does a conversation contribute to a Trending Topic?
Every completed conversation is assigned to one topic by the system. The system reads the full transcript (all messages) and picks the single best-fitting category from a predefined list. Each conversation adds one count to its assigned topic.
Note:
At this time, test conversations are not maintained separately from actual resident conversations. If a test conversation includes a number of unrelated messages, the system will evaluate it for “best fit” and assign it to the best-suited topic. Use the Refresh button in the agent widget to isolate conversations about different topics during testing.
Trending Topics = the topics with the most conversations in the selected time period.
What topics can conversations be assigned to?
Conversations can be assigned to the following topics:
Permits & Inspections
Trash & Recycling
Water & Utilities
Parks & Recreation
Roads & Transportation
Taxes & Payments
Business Licensing
Public Safety
Elections & Voting
Housing & Property
Events & Community
Animal Services
Health & Social Services
City Information & General
Feedback & Complaints
Vague or unclear inputs (like "hello" or "test") are assigned to City Information & General.
Does "trending" mean the topic is growing or spiking?
No, not currently. "Trending" simply means most frequent. The system does not compare to a previous period or detect sudden increases. A topic at the top of the list is there because it had the most conversations in the defined date range, not necessarily because it's rising.
Do all conversations count, even ones that were unresolved or spam?
Yes. Every evaluated conversation contributes to Trending Topics regardless of how it ended: resolved, unresolved, dropped off, spam, or blocked by a guardrail. The purpose of this view is to show what residents are asking about, not whether those conversations went well. (For quality insights, see Content Gaps.)
What time period is shown?
The dashboard currently shows the last 30 days. The date used is when the conversation was evaluated, which is typically shortly after it ends.
What can I do with this information?
Trending Topics help you understand what your residents care about most, so you can:
Prioritize which knowledge sources to add or improve.
Spot seasonal patterns (such as election questions being more frequent before voting deadlines).
Understand your community's most common needs at a glance.
Cross-reference with Content Gaps to see if high-volume topics also have unresolved conversations.