AI Security Researcher: How Hackclawd Works
See how Hackclawd brings AI-assisted security research into Slack, validates candidate bugs in a sandbox, and keeps human review in the loop.
Can an AI Security Researcher Work Alongside Your Team?
Development teams are shipping code faster, but security review still depends on people finding time to investigate a change, reproduce a bug, and document what happened. That gap is where automated security tools promise help - and where many fall short. A scanner that spits out suspicious patterns isn't the same as a researcher who can tell whether a finding is real.
Hackclawd is an experiment in putting an AI-powered bug bounty agent inside Slack. With an authorized program or codebase, it can investigate potential vulnerabilities, run proof-of-concept code in an isolated environment, and send the team evidence plus reproduction steps. The output stays a candidate for a human to assess; it isn't positioned as a replacement for security researchers.
This article breaks down how that workflow is set up, what sandbox validation does (and doesn't) prove, and how teams can evaluate an AI security agent without confusing automation with assurance. The real question isn't whether an agent can work all day. It's whether it can cut investigation effort without adding risk or drowning the people who own security decisions.
What is Hackclawd and how does it work?
Hackclawd is an AI-powered bug bounty agent built to run inside Slack. The premise is pretty straightforward: developers and product teams already coordinate work in chat channels, so security investigation can start where the team can actually see the request, follow progress, and discuss results.
A user mentions the agent in a channel, provides details about an authorized bug bounty program or codebase, and sets the scope. Then the agent investigates potential weaknesses across the areas described by the project, including web applications, Solidity smart contracts, and AI systems. When it finds something that looks like a candidate issue, it can assemble supporting evidence and a proof of concept, then report back in Slack with steps to reproduce what it observed.
In other words, it turns a security task into something the team can track, not a black-box scan that drops alerts into a ticket queue. A developer can see what was tested and what the agent thinks it found. Follow-up questions can happen in the same thread, and the report gives a human reviewer concrete material to inspect. That's a real difference from tools that return a score or a long alert list with little context.
Slack is the interface, not the security boundary
Putting an agent in Slack makes it easier to access - but it doesn't magically make the underlying work safe. The controls still come from the scope the team defines, the permissions the agent gets, and the environments it's allowed to touch. A Slack mention should not become "permission to test everything you own," including systems the company doesn't own.
For a real rollout, teams need a clear process for authorization. That means naming the target, defining what's in scope, recording who requested the test, and setting limits on what the agent may do. Slack can make that process convenient, but it can't replace it.
What the agent's report should contain
A useful finding needs more than a vulnerability label. Hackclawd's report (as described) includes the potential issue, evidence, proof-of-concept code, and reproduction steps. Together, those details let a reviewer check whether the behavior is reproducible and relevant to the application's threat model.
Treat the output as a candidate, not a verdict. The agent can misunderstand intended behavior, miss a precondition, or produce a proof of concept that doesn't translate into meaningful impact. A human still needs to verify the result, judge severity in context, and decide what to fix. The point is to reduce the work needed to reach that decision - not to remove the decision.
How does sandbox validation improve AI security findings?
Automated security tools often generate results that look convincing but still need a real investigation. A pattern match might point to a true flaw, a harmless implementation detail, or behavior that depends on assumptions the tool didn't check. And if developers still have to reproduce every alert from scratch, the tool may just reshuffle effort instead of saving it.
Hackclawd adds a validation step: it can run proof-of-concept code inside an isolated Upstash Box sandbox. The source describes restricted network access and execution against local copies rather than live targets. That choice matters. It's meant to let the agent show that a suspected behavior can be demonstrated - without blasting random test traffic at a production system.
A reproducible proof of concept makes a report easier to assess. Instead of asking a reviewer to guess what could happen from a snippet, it provides a concrete sequence of actions and observed behavior. That helps separate a plausible lead from a finding with evidence behind it. It also makes handoff less frustrating: the person fixing the issue has a clearer starting point than a generic warning.
A sandbox reduces risk; it does not remove it
Isolation only helps if it's actually enforced and tested. Teams should ask what the sandbox can access, how local copies are created, what data is included, whether secrets are stripped, and how execution time and resources are limited. "Restricted network access" is a good sign, but it doesn't answer every question about containment.
The local-copy approach also has trade-offs. It may be safer, but the copy might not match production configuration, identity providers, external services, or data-dependent behavior. Something that works locally can fail in the deployed system, and vulnerabilities that rely on production integrations might never show up. Validation increases confidence in a specific claim, but it doesn't prove the whole application is secure.
Proof of concept is not the same as impact assessment
A proof of concept shows an observed behavior under specific conditions. It doesn't automatically tell you how severe the issue is, whether it's reliably exploitable at scale, or what the business impact looks like. Those judgments require real context: how the system is deployed, what data it handles, and what controls surround it.
A sensible workflow keeps these stages separate:
- The agent identifies a possible issue and explains its assumptions.
- The sandbox attempts to reproduce the behavior in a constrained environment.
- A human reviewer checks the evidence and decides whether the issue is valid.
- The team assesses impact, prioritizes a fix, and verifies the change.
That separation helps avoid a common mistake: treating a technically interesting demo as a confirmed, high-priority vulnerability. It also helps avoid the opposite mistake - dismissing a useful lead because the agent can't fully model production.
What tools power the Hackclawd workflow?
The project describes four pieces: Slack for the user-facing interface, Switch to connect Slack with the AI agent and manage conversations, Nebius for the AI model, and Upstash Box for isolated proof-of-concept execution. Each component plays a different role. The hard part isn't getting each service to work on its own - it's handling the handoffs between them.
Slack is where a user starts a request and where the report comes back. Switch is described as the integration layer that connects the messaging experience to the agent and handles conversations across channels and threads. Nebius supplies the model used for investigation, and Upstash Box provides the constrained environment to execute proof-of-concept code.
This separation can also make debugging easier. If an investigation doesn't pan out, the team can ask whether the problem is request handling, model reasoning, or sandbox execution. If one component changes, the integration layer may keep the user-facing workflow stable. But every added service also adds operational dependencies - authentication, availability, data handling, permissions, logging, and failure recovery all need real attention.
The integration work is part of the security design
A security agent may process source code, vulnerability details, internal system names, and conversation history. Those are sensitive inputs. Teams evaluating an implementation should understand what information goes to each service, how long it's retained, who can access it, and whether the setup meets their contractual and regulatory obligations.
The same scrutiny applies to Slack permissions. If the agent can read every channel, it may collect far more information than it needs. A narrow channel scope and a clear request process are usually easier to govern than broad workspace access. Access should match the task, not the convenience of the initial integration.
Operationally, the system also needs clear behavior when something goes wrong. If the model times out, the sandbox can't start, or the report can't be posted, the user should see that the investigation is incomplete. A silent failure can be worse than no automation, because the team may assume a check ran when it didn't.
Measure the workflow, not the number of scans
Launching lots of investigations isn't automatically useful. Better measures include how many candidate findings end up confirmed, how long reviewers spend validating them, how often reports contain enough detail to reproduce an issue, and whether developers can fix confirmed vulnerabilities faster.
Track false positives and missed issues when you can - but interpret them carefully. A low false-positive rate only matters if the tool still covers the risks that show up in your application. Likewise, speed isn't a win if reports omit evidence or create extra triage work. The real metric is total effort and total risk across detection, validation, remediation, and verification.
How should a team evaluate an AI security agent?
Start with a narrow, authorized pilot. Don't give an agent broad access and then assume it will behave safely by default. Pick one codebase or bug bounty scope, assign a human owner, and define what the agent may inspect and execute. Keep the first run observable: reviewers should be able to see the request, the target, the testing activity, and the evidence the agent produces.
Before the pilot, agree on what success looks like. For example, the team might measure whether the agent can produce reproducible findings for a known set of issues, reduce the time spent investigating candidate bugs, or generate reports developers can act on without repeated back-and-forth. The metric should match the bottleneck you actually have. If reviewer capacity is the limiting factor, more findings may just create more work.
A practical evaluation checklist
- Authorization: Is every target explicitly in scope, and can the team show who approved the test?
- Access: Does the agent receive only the source code, channels, and credentials it needs?
- Execution: Are proof-of-concept runs isolated, resource-limited, and prevented from reaching live targets?
- Evidence: Does each report include assumptions, reproduction steps, and observed results?
- Human review: Is a named person responsible for confirming findings and assigning severity?
- Data handling: Do you know what information reaches the model and other services, and how it is retained?
- Failure states: Can users tell the difference between a completed investigation and a timeout or partial run?
- Operational value: Does the workflow reduce total review effort without flooding the team with low-quality candidates?
Decide what the agent should not do
A good pilot spells out prohibited actions as clearly as allowed ones. For instance, you might allow analysis of a local repository and sandbox execution, while forbidding production testing, destructive actions, access to customer data, or autonomous code changes. Those boundaries aren't proof the tool "failed." They're what keep experimentation governable.
There's also a real trade-off between autonomy and oversight. More autonomy can reduce manual steps, but it can also make failures harder to predict and audit. A team with mature security operations and strong isolation may accept more automation than a small company with limited monitoring. Start with human approval wherever scope, execution, or remediation could create material risk, then relax controls only when the evidence supports it.
Conclusion: automate the investigation, retain the judgment
Hackclawd points toward a useful direction for security tooling: bring investigation into an existing team workflow, then make candidate findings easier to reproduce and review. Connecting Slack, an AI model, and an isolated execution environment can reduce the friction between spotting a possible bug and giving a developer something concrete to evaluate.
But the agent's value depends on the boundaries around it. Scope has to be authorized, access has to be limited, sandbox behavior has to be understood, and reports have to remain subject to human review. A system that runs continuously isn't automatically something you can trust continuously.
If you're considering AI-assisted security research, the practical next step is a small, measurable pilot with clear limits. Keep a human responsible for the final judgment, and expand automation only when it demonstrably improves investigation quality without increasing operational risk.