Is my AI-built app safe to publish? A checklist you can run yourself
Seven checks, in the order that matters. The first four are where nearly every real problem lives; the fifth is the one no tool can do for you. None of it requires reading code.
A thing worth saying before the list: “built with AI” isn’t the problem. The problem is that the fastest path through any builder leaves a few doors open, nothing in the process closes them, and the app works perfectly either way — so there’s no moment where you find out.
Your app working is not evidence that it’s safe. It works identically whether the database is locked or wide open.
- 01
Is Row Level Security on, with policies?
Why: This is the one that actually leaks real people's data, and it's the most common finding by a distance.
How: Supabase dashboard → Table Editor. Any table showing 'Unrestricted' is readable by the public right now. Note that RLS being *enabled* isn't the finish line — a table with RLS on and a policy of USING (true) is open too.
- 02
Is there a secret key in your JavaScript?
Why: A leaked service_role or Stripe secret key hands over everything the key can do, and it's already been downloaded by everyone who opened your site.
How: F12 → Sources, or view-source and search the bundles for sk_live, service_role, sb_secret, and 'password'. Anything labelled secret, private or admin should not be there.
- 03
Can a stranger download your uploads?
Why: Storage buckets default to public in most builder templates, and profile photos and ID documents end up in them.
How: Open a file URL from your app in a private browsing window. If it loads without signing in, so can anyone with the link — and bucket listings are often browsable, which means they don't even need the link.
- 04
Is .env being served?
Why: It gets deployed by accident, and automated scanners find it within hours of a site going live.
How: Visit yourapp.com/.env in a private window. Also try /.git/config. Both should 404. If either returns content, treat every value in it as compromised and rotate.
- 05
Does anything work that shouldn't?
Why: No scanner catches this, including ours. It's the one that needs you.
How: Sign in as one test user, note an ID from the URL, then sign in as a second and try to open the first one's records. If you can see them, that's a real problem and no tool would have told you.
- 06
Do you send basic security headers?
Why: Defence in depth, not an emergency. Worth doing before you grow, not before you launch.
How: Any header-checking tool, or the Network tab: look for Content-Security-Policy, Strict-Transport-Security, X-Frame-Options and X-Content-Type-Options on the main document.
- 07
Do you know what you'd do if it went wrong?
Why: Not a technical check. If personal data leaked tomorrow, the cost is mostly determined by how fast you notice and who you tell.
How: Know where your provider's logs are, know how to rotate every key you use, and — if you're shipping for a client — know who you'd call.
What this checklist doesn’t cover
It’s a floor, not a ceiling. It won’t find flaws in your business logic, anything behind a login that only breaks under a specific sequence of actions, or a vulnerability in a dependency you didn’t know you had. A real penetration test is a person spending days trying to break your specific app, and nothing automated replaces that.
What it will do is catch the things that leak data through the front door, which is where nearly all of the actual damage in AI-built apps has come from so far. We keep a list of our own limits too.
Assay checks for this automatically, from outside your app, with no login and no access to your code — and tells you in plain English what it found, with the fix attached. What it doesn’t check is written down too.
Read next
- Why Lovable apps ship with the database openRow Level Security is off while you build because it has to be, and nothing ever turns it back on. The mechanism, a 30-second self-check, and the exact fix.
- Which Supabase keys are safe in the browserThe anon key is meant to be public. The service_role key ends your security model. How to tell them apart in ten seconds, and what to do if the wrong one shipped.
- An API key leaked into your app. What to do in the next ten minutesDeleting the key from your code does nothing on its own — the old build is still out there and the key still works. The order of operations that actually closes it.