opens in a new tab

Building a Slack sidekick — targeted CI alerts + cross-workspace invites

How we turned noisy CI alerts into targeted signals and solved cross-workspace incident invites with a single Slack app

· 8 min read

Illustration of a person untangling a chaotic knot of lines into orderly signals routed to distinct endpoints
Simple workflows can turn noise into signals

If you've spent any time as a platform engineer working with distributed systems, you have faced a distinct pain.

Distributed systems distribute everything — including the alerts.

Suddenly you've got 100 services, 100 pipelines, and 100 webhook URLs pointing to channels nobody remembers creating.

  • Ever set up a Slack webhook URL to send messages from CI and wonder if anyone actually sees the alerts?
  • Ever needed someone in an incident channel but hit a wall because they're in a different Slack workspace?
  • Ever wished CI failures went directly to the dev who broke the build instead of @channel?

We built a Slack app to help with this.

Let's call it Sidekick — a lightweight internal tool that routes CI/CD notifications to the right humans and invites anyone from multi-workspace slack org into incident channels without the friction.

Here's how we did it, the scopes we used, and the lessons we learned along the way.

The Problem with Webhook-Based Alerts

The traditional setup for CI/CD alerts looks something like this:

  1. Create a Slack channel (#build-alerts, #deploy-notifications, etc.)
  2. Generate an incoming webhook URL
  3. Paste that URL into your CI configuration
  4. Watch as alerts flood in
  5. Watch as your team develops webhook blindness

The fundamental issue? Webhooks are channel-centric, not people-centric. When a deployment fails, the webhook screams into a channel. The developer who actually broke the build? They might be in a meeting, have notifications muted, or simply not be watching that channel at all.

Even worse: have you ever tried to figure out where a Slack webhook URL is actually posting? It's like trying to find a needle in a haystack, except the haystack is your CI configuration spread across 47 repositories and the needle might actually be pointing at a channel that was renamed six months ago.

We needed something better.

What We Actually Wanted

  • Direct messages to the responsible developer when their commit breaks something
  • Channel messages that work without hunting for webhook URLs
  • A single integration point instead of dozens of webhook configurations
  • GitHub-to-Slack user mapping so we can route alerts based on who made the commit

Building the Bot Foundation

The first step was creating a Slack app with the right bot permissions. Here's what we needed and why: 

# Bot Scopes
chat:write
chat:write.public
channels:read
users:read
users:read.email
incoming-webhook
channels:write.invites

With these scopes, we could:

  1. Receive alerts from CI through a simple HTTP endpoint
  2. Map GitHub emails to Slack users using the Users API
  3. Send targeted DMs to the developer who caused the issue
  4. Post to channels without managing a constellation of webhook URLs

The Architecture

Our setup is straightforward:

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   CI Pipeline   │────▶│  Sidekick API   │────▶│   Slack API     │
│ (GH Actions etc)│     │   (Internal)    │     │   (Bot Token)   │
└─────────────────┘     └─────────────────┘     └─────────────────┘
                              │
                              ▼
                        ┌─────────────────┐
                        │  User Mapping   │
                        │  (email→slack)  │
                        └─────────────────┘

The CI pipeline sends a payload to our internal API containing:

  • The message to send
  • The GitHub committer's email (or username)
  • Optional: a channel or webhook url to post to, severity level, etc.

Sidekick API looks up the Slack user ID from the email, then sends a direct message. No more hoping someone is watching the alerts channel!!

Backward Compatibility with Webhooks

Since we included the incoming-webhook scope, teams with existing webhook integrations could keep using them while migrating. We essentially created a webhook-compatible endpoint that accepted the same payload format but routed messages through our smarter logic.

This was crucial for adoption-nobody wants to rewrite all their CI configurations on day one.

Beyond Single Messages — Interactive Workflows

Once you have a bot that can DM developers, you start seeing other opportunities. What about those CI/CD workflows that require manual approval gates?

Instead of a notification that says "Deployment waiting for approval, check Github Actions(or Codefresh or Jenkins)," you can send an interactive message with Slack Block Kit: 

{
  "blocks": [
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "*Production deployment ready*\nCommit: `abc123` by @developer\nService: payment-service"
      }
    },
    {
      "type": "actions",
      "elements": [
        {
          "type": "button",
          "text": { "type": "plain_text", "text": "Approve" },
          "style": "primary",
          "action_id": "approve_deploy"
        },
        {
          "type": "button",
          "text": { "type": "plain_text", "text": "Reject" },
          "style": "danger",
          "action_id": "reject_deploy"
        }
      ]
    }
  ]
}

Engineers can approve or reject directly from Slack without context-switching to another tool. This is where the bot starts becoming a real productivity multiplier.

The Enterprise Grid Challenge

Everything we built so far works great within a single Slack workspace. But then reality hits: Enterprise Grid.

(Cue dramatic music.)

If your organization uses Slack Enterprise Grid, you likely have multiple workspaces-maybe one for employees, another for contractors, separate ones for different business units, or legacy workspaces from acquisitions that nobody had the heart to merge.

When an incident occurs and you need to spin up an incident channel, you want everyone relevant in that channel. But here's the catch: a bot installed in Workspace A cannot invite users from Workspace B.

This is where most teams hit a wall. The usual "solution" involves:

  • Requesting Channel Management Tools access
  • Getting IT approval
  • Filling out a ticket in triplicate
  • Sacrificing a small goat to the access management gods
  • Waiting while your incident burns and your MTTR weeps

We wanted our Sidekick to handle this automatically.

User Scopes for Cross-Workspace Operations

To operate across workspaces in an Enterprise Grid, we needed user token (not just a bot token) and the app needed to be installed at the organization level: 

admin.conversations:read
admin.conversations:write
admin.users:read
channels:read
users:read
users:read.email
chat:write

The key difference: bot tokens are workspace-scoped, but user tokens with admin scopes can be org-scoped when installed when the app is installed at the organization level.

Enabling Organization-Level Installation

For your app to be installable at the org level (not just individual workspaces), you need to enable it in your app's settings: 

{
  "settings": {
    "org_deploy_enabled": true
  }
}

This tells Slack that your app can be installed to a Slack organization.

OAuth Setup for Org Installation

When you enable org-level deployment, you need a proper OAuth flow. The user who installs your app (typically an Enterprise Grid admin or owner) will go through an authorization flow that grants your app the admin scopes across the organization.

Here's what the flow looks like:

1. Admin clicks "Install App to Org" from your app's OAuth page
   ↓
2. Slack redirects to: https://slack.com/oauth/v2/authorize
   (with your client_id and requested scopes)
   ↓
3. Admin authorizes the app and grants org-level permissions
   ↓
4. Slack redirects to your callback: https://your-server.com/slack/oauth/callback?code=...
   ↓
5. Your server exchanges the code for access tokens
   ↓
6. You store the user token securely—this is your user scoped auth token

For detailed implementation guidance, check out Slack's official documentation: Installing with OAuth

The important pieces:

  • Register a redirect URL in your app configuration
  • Implement the /slack/oauth/callback endpoint to handle the code exchange
  • Securely store the resulting user token (this has admin permissions!)

A Critical Gotcha: If your app was previously installed in a workspace at the workspace level, and then you enable org-level deployment and try to install it at the org level, you may encounter a confusing error:

"This app is requesting more permissions than what your org admin has granted for your organization"

After chatting with Slack support, we learned that some workspace-level scopes conflict with org-level scopes. The cleanest solution is to create a fresh app that has never been installed in any workspace, configure it for org deployment from the start, and install it at the org level first.

This cost us a few hours of debugging. Learn from our pain.

Adding a Slash Command

With cross-workspace user invites working, we wanted to make it easy for anyone to use. Enter the slash command: /incident-invite.

Because if there's one thing engineers love, it's typing commands. We're basically just building CLIs for Slack at this point.

When someone types /incident-invite @username in an incident channel, our Sidekick:

  1. Receives the command payload at our API endpoint
  2. Looks up the mentioned user across all workspaces
  3. Invites them to the current channel (even if they're in a different workspace)
  4. Confirms the invite in the channel

Setting Up the Slash Command

In your Slack app configuration, add a slash command with:

  • Command: /incident-invite (or whatever fits your naming convention)
  • Request URL: Your API endpoint that handles the command
  • Description: A helpful hint like "Invite a user from any workspace to this channel"

Remember this will add the commands scope to your bot scopes to receive slash command payloads and if already installed, you have to re-install the app.

The Complete Manifest

Here's the complete app manifest you can use as a starting point. Copy, paste, update the URLs, and you're 80% of the way there. (The other 20% is, as always, "everything else.") 

{
  "display_information": {
    "name": "Sidekick Concierge",
    "description": "Invite humans across workspaces into slack channels on demand, push signals to slack users!",
    "background_color": "#0a0a09"
  },
  "features": {
    "bot_user": {
      "display_name": "Sidekick Concierge",
      "always_online": true
    },
    "slash_commands": [
      {
        "command": "/incident-invite",
        "url": "https://your-server.com/api/slack/command",
        "description": "tag a user (cross workspace) to be invited",
        "usage_hint": "@username",
        "should_escape": true
      }
    ]
  },
  "oauth_config": {
    "redirect_urls": ["https://your-server.com/slack/oauth/callback"],
    "scopes": {
      "user": [
        "admin.conversations:read",
        "admin.conversations:write",
        "admin.users:read",
        "channels:read",
        "users:read.email",
        "users:read",
        "chat:write"
      ],
      "bot": [
        "channels:read",
        "channels:write.invites",
        "chat:write",
        "chat:write.public",
        "incoming-webhook",
        "users:read",
        "users:read.email",
        "commands"
      ]
    }
  },
  "settings": {
    "org_deploy_enabled": true,
    "socket_mode_enabled": false,
    "token_rotation_enabled": false
  }
}

What You Can Build From Here

Once you have this foundation, the possibilities expand:

  • Restrict slash commands to specific roles: Only incident commanders can use /incident-invite
  • Audit logging: Track every cross-workspace invite for compliance
  • Channel templates: Auto-create incident channels with predefined members
  • Integration with PagerDuty/OpsGenie: Automatically invite on-call engineers
  • Smart routing: Send different severity alerts to different channels/people
  • Acknowledgment tracking: Know when someone has seen and acknowledged an alert

The key insight is that by centralizing your Slack operations through a single, well-scoped app, you gain control and visibility that's impossible with scattered webhook URLs and manual processes.

Summarized Learnings

  • Bot scopes first, admin scopes sparingly. Only escalate to admin/user scopes when you genuinely need cross-workspace operations.
  • Design for Enterprise Grid early. Retrofitting is painful. Ask us how we know.
  • Guard your admin tokens. They're powerful. Treat them like production database credentials.
  • Support legacy webhooks during migration. Your teammates will thank you (or at least not curse you).
  • Interactive messages pay dividends. Buttons beat "check CI" every time.

What started as a solution to noisy CI alerts evolved into a Swiss Army knife for Slack operations. Our Sidekick now handles:

  • Targeted CI/CD notifications that reach the engineer, not just a channel
  • Cross-workspace incident invites without friction
  • Interactive approval workflows for deployments and other gates
  • Unified message routing that replaces a mess of webhook URLs

Building this required understanding Slack's permission model-specifically the difference between bot tokens and user tokens, workspace scopes and org scopes. But once you grasp those concepts, you can build tooling that dramatically improves your organization's incident response and developer experience.

The best part? Everything we built runs on a single, straightforward Slack app and a simple express server. No complex infrastructure-just a well-scoped app and an API that knows how to use it.

Now go build your own Sidekick. Your future on-call self will thank you.

Ink tailpiece: Vignesh, his wife, and Chocolata standing together on a single brush stroke, a faint sunset on the water behind them.