How Ramp Killed Its Data Bottleneck With an AI Slack Bot

The Problem

Most teams eventually hit the same wall - everyone has an ad hoc data question that gets funneled through one or two people, creating a huge bottleneck. Dashboards aren’t the fix, because dashboards only answer the question that someone already thought to build. And the big risk goes past just the wait time. Most data questions just stop getting asked because it means waiting in a queue.

Ramp hit this roadblock and wrote about it in their engineering blog. Before they had Ramp Research (their internal Slack-based data agent), the company routed data questions through a single on-call analyst. In Ramp's own words: "decisions wait and most questions go unasked."

The System

Someone @mentions the bot in Slack with a question in plain English. The bot doesn't have a pre-built list of answers. It decides, per question, what to query. It writes SQL against a real database, runs it, and replies with both the answer and the exact query that produced it.

Ramp Research posts CSV previews in-thread so people can validate results themselves rather than take the bot's word for it. I found the same for my build, where I was hesitant to trust the LLM. So I decided to add a preview of the query used that can be validated by a human.

For the dataset, I decided to use NYC's public Citi Bike trip data, since that’s a large enough (and interesting enough) dataset that I can work with.

The Build

The mechanism is a Claude tool-use loop, not a static prompt stuffed with data. Claude gets one tool, query_database, and decides for itself what SQL to run based on the question asked.

while True:
    response = client.messages.create(
        model=MODEL,  # claude-haiku-4-5 — cheap and fast enough for SQL writing
        max_tokens=1024,
        system=SYSTEM_PROMPT,
        tools=TOOLS,
        messages=messages,
    )
    if response.stop_reason != "tool_use":
        text_blocks = [b.text for b in response.content if b.type == "text"]
        return "\n".join(text_blocks).strip(), executed_sql
    # ...execute the tool call, append the result, loop again

SYSTEM PROMPT is where the ambiguity handling and guardrails live:

SYSTEM_PROMPT = f"""You are a data analyst answering questions about NYC Citi Bike ride data in Slack.

{SCHEMA}

You have one tool, query_database, which runs a read-only SQL SELECT against this table. Always answer using real numbers from a query result — never guess or estimate without querying first. Keep answers short and Slack-friendly: a sentence or two, with the key number(s) called out.

If the question is ambiguous (e.g. "busiest" could mean most rides started, most rides ended, or most total activity; a time range like "last week" isn't explicit about which week), briefly state how you interpreted it as part of your answer — don't ask a clarifying question and wait, just say what you assumed so the person can immediately tell if that's what they meant and re-ask if not."""

I made sure the LLM knows how to handle ambiguity, instructing it to state its assumptions explicitly, so the user knows the definition of “busiest”, for example.

I also added a safety boundary that I caught after my initial review. This will help prevent any rogue deletions or insertions from the bot.

def run_query(sql: str) -> list[dict]:
    stripped = sql.strip().rstrip(";")
    if not stripped.upper().startswith("SELECT"):
        raise ValueError("Only SELECT queries are allowed.")
    if ";" in stripped:
        raise ValueError("Only a single statement is allowed.")

The first version only checked that the query started with SELECT, so "SELECT 1; DROP TABLE rides;" would have passed that check and run both statements. I quickly realized a prefix check isn’t a safety boundary.

The dataset was 3.86 million April 2026 Citi Bike trips, which was too large for a free-tier Postgres instance, so the build loads a random ~1 million-row sample instead. But this system can scale infinitely.

Here’s an example of what the system looks like in action.

A basic question to see the “busiest” stations (shoutout to Chelsea):

A question that asks the bot to make assumptions (which it flags and denies):

Tradeoffs & Exclusions

Slack already ships a native AI agent. As of Salesforce's March 2026 update, it's an MCP client, meaning it can reach external tools and data sources.

So at first I thought a build like this isn’t worth it, but there’s a catch. Getting it to answer questions against a specific database still requires building an MCP server exposing that data, which is the same core engineering work as this build, layered on top of a Business+ or Enterprise Slack AI subscription (which this build doesn't need).

Ramp shipped Ramp Research in August 2025, before Slack's MCP capability existed at all. Timing alone may explain the choice, but I assume a fintech routing internal financial queries through a third-party agent platform would have influenced the decision towards the same outcome either way.

This build runs on Slack's free plan, does one thing, and nothing about it depends on a vendor's roadmap.

There is also the choice of a cloud API over a local model. Anthropic's API doesn't use inputs or outputs for training by default, and deletes them within 30 days under standard retention (allegedly). Zero Data Retention also exists, but as an enterprise sales agreement.

For Citi Bike's public data, that's a non-issue. For the build this one stands in for, a bot answering questions about revenue, churn, or customer data, every query result leaving the company's infrastructure on its way to an answer is a governance question. The alternative, a local or open-weight model, keeps everything on owned hardware and trades away reasoning quality plus the operational cost of hosting inference in-house.

What's deliberately not in this version: multi-turn memory. In the current state, the bot can't connect a follow-up question back to the one before it. One consequence of that is handled: an ambiguous question like "busiest" (by rides started, ended, or total activity?) or "last week" (since when, exactly?) doesn't get a clarifying question and a wait, since without memory the bot couldn't connect the answer back to the original question anyway. Instead, it states its interpretation inline, "measuring by ride starts in the last 7 days of the dataset," so a wrong guess is visible immediately instead of being silent.

Schema self-discovery is also cut: the bot only knows about the columns it's told about in its system prompt, written by hand. If you add a table or rename a column, the bot has no idea until someone edits that prompt.

The fix is two separate mechanisms. Querying Postgres's own information_schema handles structural accuracy, since it's always exactly correct: it is the live schema. A single current-state schema file handles the semantic layer instead- why a column exists, whether it's deprecated- rather than a folder of incremental migrations, which would mean replaying history the live database already reflects.

Measuring Success

Ramp's own numbers are the industry benchmark here. Over 1,800 data questions across 1,200 conversations with 300 users in the weeks after launch (damn) and a self-reported 10 to 20x increase in questions asked.

Future State

The nearest fix is the schema problem above: information_schema querying plus a current-state schema reference, so the bot stays correct as the data changes instead of drifting.

Multi-turn memory is the next step after that, so you can ask clarifying questions and not just get back an inline restatement.

Past that, the improvements get a bit more operational. For example, tracking what this costs per question. A system with usage-driven cost should probably have cost observability.

Nothing scopes who can ask what. So anyone in the channel gets identical access to all the data, which is fine for public bike-share numbers but not for revenue data (for example).

None of this is Citi Bike specific. It’s straightforward to swap the connection string and the schema description, and the same mechanism answers questions about any Postgres-backed data a business already has.