Building AI Powered Refunds

_{written by Alex Berman, Software Engineer at Rec and Elise Hudson, Product at Rec}

What are AI refunds?

AI refunds bring artificial intelligence into one of the most judgment-heavy workflows for recreation departments: processing cancellations and issuing refunds. By reading policies, interpreting context, and learning from past decisions, an AI system can recommend the right outcome.

The difference from traditional rule-based automation is adaptability. Instead of following rigid “if this, then that” rules, AI refunds can recognize patterns, account for exceptions, and approximate the decisions staff would make themselves.

Where did the idea come from?

Refunds are deceptively complex. Policies often stretch across pages of text, but decoding a refund policy could be a full-time job, and real-world practice rarely follows them word for word. Staff exercise judgment - choosing credit instead of refund, waiving fees, or applying flexibility when necessary. We wanted to help staff reduce time searching through policies and manually calculating refunds so they could spend more time on other tasks. So we set out to build our AI Refund Agent.

‍

What We’ve Learned so Far

We’re lucky to work with forward-thinking partners who are eager to test and learn with us. The City of Torrance was especially open in this process, letting us do a deep dive into their refund policies and historical refund decisions. That partnership gave us the chance to see not just how policies are written, but how they are actually applied day-to-day.

Here’s what we observed when we launched the feature:

‍

AI can have a huge impact on repetitive work

We've tested our recommendations against 1000s of historical refunds, and almost immediately, 50% of our AI’s suggestions were exact matches, down to the recommended amount and refund method. We're excited at just how powerful and accurate AI can be. In these cases, we can automate the process end to end, dramatically reducing the time spent on manual tasks.

‍

Often, there's more than one "right" answer

In another 30% of recommendations, the AI made the right policy decision, but in the real world, a staff member chose to do something else. For example, a staff member might offer a cash refund, even if the policy states credit only. This revealed something important: written policies rarely capture the operational preferences that develop over time. Organizations make choices about when to prefer credits, how flexible to be around cancellation cutoffs, and how to handle specific program types. To build great AI features, we believe it's important for the city to be able to trust the judgement of their AI counterparts. This meant making sure that AI suggestions were fully auditable, and any staff member could dig into the logic for any suggestion made by the AI.

In AI world, we call this "human in the loop" design, which is a core principle of every AI feature in Rec. By exposing the "thinking" of AI, we enable staff to build trust our features by reviewing its work, just as you would with a new employee that joins your team.

Simpler inputs dramatically reduce error rates

Early in testing, 13% of recommendations were flat-out wrong, often because the AI latched onto irrelevant policy details like check-mailing timeline or contact instructions. We found that clearly written, concise policy, enabled the model to replicate real world results more reliably. So we worked with our pilot cities to strip out unneccessary elements from an AI-specific policy, including only what matters for refund decisions. This simplification not only prevents confusion but also helps avoid “context rot,” where long policy inputs cause models to treat earlier information with more reliability than later sections. Through this process, we cut the error rate to 3%!

Finally, feedback is essential - and staff expertise is irreplacable.

Staff bring years of accumulated knowledge about edge cases, exceptions, grace periods, and unwritten norms. Their feedback helps us understand where policies diverge from real practice and where our recommendations need refinement. On every new refund suggestion, we enable staff to let the agent know if their suggestion was correct. This "trains" the model over time to understand how a city would react in unique scenarios. As this feature rolls out, those insights allow us to use feedback to bring best practice analysis to everyone, and also make the agent more adaptable to each organization’s workflows.

What else can AI Refunds unlock?

Improving refund recommendations is only the beginning. As AI workflows get smarter, they can open new opportunities for departments:

1. More concise policies.
   Policies often include details that matter for humans (like check mailing timelines) but only add noise for automation. AI can highlight which parts of a policy impact refund eligibility and which don’t, helping departments write policies that are easier to understand and enforce.
   
   
2. Insight into when policies aren’t followed.
   In Torrance, 17% of cancellations were handled outside written rules, but in ways that made sense for staff and residents. By surfacing these patterns, AI can give leaders visibility into when and why flexibility is exercised, turning “exceptions” into useful knowledge.
   
   
3. Adaptable recommendations.
   Policies tend to be rigid (“refunds must be requested 7 days in advance”), but real life is messier. AI can recognize grey zones, like cancellations 6 days before an event, and flag that, for example, admins still issue credits in 40% of those cases. That makes recommendations not just accurate, but adaptive to how policies are applied in practice.
   ‍
4. Proactive Action.
   Imagine a rainy day. You have to cancel 100s of picnic reservations. The refund agent can immediately step in and process all of those refunds according to policy, instead of waiting for customers to ask or going reservation by reservation and making tedious calculations.

We’re excited to have launched AI refund suggestions across multiple cities, and we’ve recently rolled out the customer-facing experience, which warns users before they cancel if they’re likely to receive a refund and displays the relevant policy text. More to come soon!