At RapDev, we are always looking for ways to make ServiceNow work smarter–not just faster. One of the most persistent pain points we see across the organizations we work with is the quality of incident resolution notes. Technicians under pressure are rushing to close tickets, and the resolution notes that follow are often vague, inconsistent, or so generic they offer zero value to the next person who runs into the same problem. "Resolved issue" or "Rebooted and it works now" might close the ticket, but they do nothing for the knowledge base, the customer, or the team.
We recently set out to solve that problem using Now Assist–and what we built goes well beyond a simple AI prompt.
The Problem: Resolution Notes Nobody Reads (Because They Say Nothing)
Incident resolution notes are supposed to capture what went wrong, what was done to fix it, and ideally why the fix worked. In practice, most organizations see a wide spectrum of quality, from engineers who write detailed, structured resolutions to those who leave a one-liner that tells you nothing.
This inconsistency hurts:
- Knowledge reuse: The same issue gets solved from scratch next time
- Customer communication: Vague notes mean vague closure summaries
- Reporting & trending: If you can't describe what you fixed, you can't identify patterns
The goal of this project was to close that gap using AI–but with guardrails. We didn't want to just generate a resolution and call it done. We wanted to assist the technician and validate that the AI-generated content was actually accurate and specific to the incident at hand.
The Solution: A Two-Stage Now Assist Pipeline
What we built is a two-stage AI pipeline living entirely within ServiceNow, powered by Now Assist skills and agents.
Stage 1: Resolution Notes Template Population
The first stage activates when a technician is ready to document their resolution. Rather than staring at a blank text field, they trigger a Now Assist skill that does the heavy lifting of pulling context from the incident itself.
The custom skill, inspired by ServiceNow’s out-of-the-box Incident Summarization skill, reads the incident's short description, description, and comments and work notes–the running log of everything the technician did, tried, and discovered throughout the lifecycle of the incident. It then synthesizes that context and populates a structured resolution notes using a provided template which is saved in a system property.
The template is intentionally opinionated. It guides the AI–and by extension the technician–toward capturing:
- The specific error or symptom that was reported
- The root cause that was identified
- The exact steps taken to resolve it
- Any relevant context that would help someone reproduce the fix
By the time the technician reviews the pre-populated note, they already have a strong first draft. Their job becomes reviewing and refining rather than writing from scratch.
Stage 2: AI-Powered Quality Validation
Here is where the project gets interesting. When the technician saves the record, a Now Assist agent is triggered automatically. This agent is equipped with two distinct skills that work in tandem to validate the quality of the resolution note before the ticket is considered truly closed.
Skill 1: Specificity Analysis
The first skill performs a contextual comparison between the resolution note and the original incident data–specifically the short description and description. Its job is to answer a simple but critical question: Does this resolution actually address the specific problem that was reported?
This guards against a common AI failure mode where a generated response sounds reasonable but drifts from the actual issue. If a technician reported a login failure tied to an expired certificate, the resolution should reference that certificate–not offer generic advice about clearing cache and rebooting. The skill flags resolutions that are technically correct but not contextually matched to the incident.
Skill 2: Quality Benchmarking Against Golden Standards
The second skill takes quality validation a step further by comparing the resolution note against a curated library of golden standard examples–real, high-quality resolution notes that represent what "great" looks like in your organization.
Rather than evaluating the resolution in isolation, this skill checks whether it meets the same bar as the best documentation your team has produced. It flags broad, vague language–the "applied fix and resolved" style of writing–and scores the resolution against the patterns observed in the exemplary set. This creates a feedback loop that continuously rewards specificity and penalizes ambiguity, reinforcing good documentation habits across the entire team.
What It Took to Build It
Getting this pipeline to production required work across multiple layers of the Now Assist platform.
On the skill development side, we authored and configured custom Now Assist skills for both the template population and the two validation functions. Each skill required careful prompt engineering to ensure the AI understood not just the task but the standard it was being held to–particularly for the golden standard comparison, where framing the benchmarking context correctly made a significant difference in output quality.
On the platform configuration side, we built the agent orchestration layer that coordinates the two validation skills, handles the trigger logic tied to the record save event, and manages the feedback surfacing so technicians get actionable, non-disruptive guidance when their resolution doesn't pass muster.
We also invested time curating the golden standard library–identifying and tagging exemplary resolution notes from historical incident data, then building that corpus into the skill's context in a way that gave the AI meaningful reference material rather than noise.
Throughout development, we iterated heavily on the specificity analysis skill. Getting the model to distinguish between a resolution that is technically accurate and one that is contextually accurate to this incident required several rounds of refinement. The final approach involves anchoring the skill's evaluation directly to extracted entities from the short description and description fields, rather than evaluating the resolution in the abstract.
The Value Delivered
The impact of this work is felt at multiple levels.
For the technician, the burden of documentation is dramatically reduced. They no longer start from a blank slate, and they get real-time guidance when their draft doesn't meet the mark–rather than finding out weeks later during a quality audit.
For the team and the knowledge base, every closed incident now carries a resolution note that is specific, contextualized, and benchmarked against the best documentation in the organization. The consistency compounds over time, making search and reuse significantly more effective.
For leadership and process owners, the quality of incident documentation is no longer dependent on individual habits or under-pressure shortcuts. The AI acts as a silent quality coach embedded directly in the workflow, raising the floor for everyone. Reporting on incident resolution quality improvement tasks can help to determine which teams or individuals are not providing quality resolutions, driving action and accountability across the organization.
And perhaps most importantly–this was built entirely within the Now Assist framework, meaning it lives where the work already happens. There is no external tool to adopt, no separate review workflow, and no friction added to the close process. The AI meets the technician where they are, at the moment that matters.
What's Next
This project lays the foundation for a number of natural extensions–automatically surfacing similar past incidents at resolution time, feeding resolution quality scores into performance dashboards, or using the validation pipeline to identify recurring root causes that suggest a deeper problem worth a problem record. The infrastructure is in place. The next step is deciding how far to take it.
If your organization is wrestling with incident documentation quality–or if you are curious about what Now Assist can do when it is paired with thoughtful agent design– reach out to RapDev. We would love to talk.
