Smashing Security podcast #480: This is the AI service you should never sign up to

You have a full stop. You have a slash. A colon. Have I said? He said colon.

Hello, hello, and welcome to Smashing Security, episode 480. My name is Graham Cluley.

I am currently the Chief AI Security and Ethics Officer at NovStar Intelligence, which is a new gig for me since I last was on this show.

I’ve also got another new podcast, which finally came about. I know we discussed that on my last episode, but it’s here. It’s here.

Not the same as this one, but variety is the spice of life, right?

It’s called Tech Film Noir, and on it, me and two other techies review the technology in science fiction films to see if it predicted the future.

And the latest episode, which will be out when this episode airs, is for Strange Days, which is a 1995 classic, in my opinion, that no one has ever heard of.

It’s all about VR and living other people’s memories and stuff. It’s quite dark, very cyberpunk, but very fun.

I am currently finishing off my master’s in AI, and for my dissertation project, I have been building synthetic incels.

The reason being is because I just can’t get hold of any real ones. Incel stands for involuntary celibate.

It’s a subgroup of the manosphere that believe that they’re incapable of having romantic or sexual relationships because society is against them.

Incels are an issue because they come with a lot of negative violence and danger.

And compared to the rest of the male population, which has a suicide rate of about 3%, incels tend to have a suicide rate between 43% and 62%.

And they’re also seeing a lot of incels use things like AI companions and AI girlfriends to enact violent tendencies.

So I thought if incels are using AI companions to practise and rehearse negative interactions with women, how about we get that AI companion to challenge these beliefs?

So I’m doing a thing where this AI companion, if it detects this incel is starting to veer off into radicalisation, it will then start challenging those beliefs.

So, getting people to reflect back on the reason why they think that.

And the results are that it does slowly de-radicalise these incels. Now, you’re probably thinking, how did you make an incel?

Well, unfortunately, I had to troll through absolutely loads, and there is loads of datasets out there of 4chan, Reddit subposts.

This week on Smashing Security.

We won’t be talking about how a survey has found many European firms are worried about the US government cutting off their access to cloud services, ranking it roughly on a par with worries about ransomware attacks.

And we won’t even mention how the United States has sentenced a Belarusian man to 16 years in prison for running the Ransom Cartel ransomware operation.

So, Lianne, what are you going to be talking about this week?

All this and much more coming up in this episode of Smashing Security.

They’ve just published a new report, 2026 State of the Cybersecurity Attack Surface, and they analysed over 800,000 real IT assets to find out how exposed organisations actually are.

Nobody added him, nobody removed him, and he’s been quietly in there for 11 years downloading maps of Paraguay.

That’s the path of least resistance.

And the report is free to download.

What would you do if someone offered you access to Claude, which is of course Anthropic’s flagship AI model, for about 90% off the official price?

I mean, that sounds pretty good, doesn’t it? Would you be tempted?

And carry on as normal. And you get the same responses, you get the same capability, you get a fraction of the cost. Pretty neat, eh?

Boffins at Okta have published a report describing just such a setup, and there are some what you may call catches. So, this service is called Poison Claude.

I was thinking you either have to admire them for their refreshing honesty by being blatant that this is something maybe a little bit toxic, or just think this is awful marketing at work.

Either way, that is what they have called it. And about 900 people so far have actually chosen to sign up for it.

We have a poisoned version. Ooh, dangerous.

I’m going to set myself up on another server and charge less, and that will be better for humanity. Maybe that’s what Anthropic is up to this week. Anyway, let’s find out.

We all know AI can get a teensy bit expensive. So if you’re accessing Anthropic’s AI models, the likes of Claude, Opus, Sonnet, etc., you’re charged by the token.

And if you use them a lot, the costs can quickly add up. And I don’t know about your own AI use, Lianne, but a lot of these things you can sort of auto-top up.

So when they’ve run out, they’ll just throw another $50 in, throw another $50 in, and before you know it, you’ve got a stonking bill by the end of the month.

They don’t take that at these frontier models. They don’t take that as payment, unfortunately. I’ve tried.

So what Poison Claude offers is the same access to Anthropic for about, well, between 5 to 15% of the normal official price, depending on which model you want.

So the question is, of course, how do they manage that? How do they get such cheap access to Anthropic? And well, the answer is there for anybody to see.

If you go to Poison Claude’s website, they explain how they do it. So Amazon runs the AWS cloud computing platform, right? It’s used by every Thom, Dick, and Harry.

Every business is using AWS. And through a part of AWS called Bedrock, they host Anthropic’s Claude models.

So instead of going directly to Anthropic, developers can access Claude via Amazon instead.

So if you sign up for a new AWS account, Amazon can give you $100 or so to spend on some of their services. So they give you some, basically like a coupon.

Every new account starts with up to about $200 worth of free Claude usage, courtesy of Amazon. Very, very nice.

And well, $200 worth of Claude usage is nice, but it’s not life-changing.

So they can collect multiple lots of $200 worth of tokens every time. And if they do that 100 times, they’ve got $20,000 worth of free Claude access.

If they do it 1,000 times, they’ve got $200,000 worth of Claude access. And that is exactly what they appear to be doing.

So Poison Claude created a factory of fraudulent AWS accounts. They collected all the free credits from each one. They pooled them all together.

They then used that pile of stolen credit to answer their customers’ AI prompts.

And when that account runs dry, the system moves to the next one. So when you pay Poison Claude 5 to 15% of the normal price to use Claude, you’re not really getting a discount.

What you’re doing is you’re getting Claude access that Amazon is unknowingly paying for. Through credits that Amazon gave to people who don’t actually exist.

They may not have realised quite the scale of this.

By the way, it’s important to point out here, it’s not as though the people who are signing up for this thought that everything was legal and above board, right?

This is a cybercriminal website and they were paying for these accounts with cryptocurrency. Now, friend of the show, Rory Keflin-Jones, he famously offered a regular reminder.

He said cryptocurrency does in fact have a use case and that use case is crime.

Anyway, if you did sign up, your prompts are going to Poison Claude. Poison Claude is passing them to the real Claude, and the answer comes back and you save yourself some money.

And this isn’t just a problem for Anthropic.

There’s a very similar operation called EcoMagent that’s doing a similar trick, offering AI startups up to $350,000 worth of credits with Google Cloud.

And now, you know, some people are going to say, stealing from Amazon and Google, does it really matter? They can afford it. They’ve got heaps of money, but it is still fraud.

And crucially, remember that you are not sending your AI prompts to Claude. You are sending them to Poison Claude first, and Poison Claude can see everything.

So the boffins at Okta, they’ve described how everything you type, every question you ask, every piece of code you paste in, every document you share — it’s like that Sting song, Every Move You Make.

Poison Claude is watching. I don’t know what’s worse, having Sting watching you or having Poison Claude watching you.

But who knows what Poison Claude is gonna do with this information, right? Maybe they will accidentally leak it. Maybe they will sell it.

Maybe they might use it to train competing AI models.

So there’s a real danger here, isn’t there?

So why are people signing up for something like this?

Because we’re so distanced from the victim, in this case Anthropic, we’re so distanced from them. And we also see them as these massive companies with massive revenues.

My little tiny bit of crime is not going to impact them.

And so we rationalise that, and that’s what enables us to be quite accepting, especially if you look around and go, well, everyone’s kind of doing something similar.

You know, there’s lots of people I know that are doing that, so it makes it feel more acceptable to do so. So I think there’s that kind of rationalisation.

It’s like, well, you know, they are charging a lot and I should have access to this. And, you know, I am paying someone to do something.

But if it’s a big global multinational who maybe isn’t paying enough tax, then you might think, well, what’s the harm in this?

I must say, that’s when I was a lot younger and had very little cash.

Things like code containing API keys, business plans, legal documents, medical queries.

I’ve got this awful rash on my left thigh — I don’t, by the way, before any AI scoops this up from the transcript.

And of course, you are trusting Poison Claude to have good security, but guess what, Lianne?

So researchers were able to run a simple command against an unprotected API, and they got back a response which showed the total number of users, exactly how many of them were active at the time.

And although Poison Claude had attempted to shield all of their servers and their locations via Cloudflare, they did leave their API endpoint exposed, which I don’t know if you’ve ever left your endpoint exposed.

It can be very painful, I can tell you.

Cloudflare has responded in their traditional style and decided it doesn’t warrant any further action, which of course —

So if you are a developer who’s been tempted by one of these cut-price AI services, perhaps you should ask yourself instead if that few quid which you are saving really is worth handing a stranger all of your secrets.

And of course, if you’re inside a business, you want to consider as well what kind of rules and controls you have in place, whether you’ve actually told your staff, this is the AI we work with, this is how you interact with it, to not do any shadow AI stuff, which maybe is unregulated and could actually lead to company secrets being breached as a consequence.

NordLayer is a network security platform built for businesses.

It’s a scary world out there for travelling workers.

But it goes well beyond just encrypting the connection.

You get centralised control over who can access what based on their identity, their device, whether their device is actually compliant.

And if someone leaves the company, you revoke their access immediately.

So if someone on your team has started using some AI tool that your security team hasn’t approved.

Use the code NLSUMMER26 at checkout.

But if you got the opportunity to name a phishing kit, what would you name it?

But today, dear listeners, we’re going to be talking about a phishing kit called Greatness.

The teams always go away and they discuss what they’re going to call their team. And their team is always like Inspire or Confidence or Team Greatness. It’s just a terrible name.

They should have called it something like Lumpy Trousers, you know, have a little bit of a sense of humour or something.

So Greatness looked at that and went, that seems like a lot of work. Why don’t we just send people to Microsoft’s actual login page instead?

And it’s really horrible because it doesn’t rely on fake websites, any kind of dodgy URLs or anything like that. Instead, it weaponises a perfectly legitimate Microsoft login flow.

And it’s a real thing that Microsoft uses for devices that can’t easily display a login screen.

So think about, you know, when you go to a hotel and you want to log into YouTube on the hotel, then you can put a code in instead, right?

Or think of conference room kits or any kind of other IoT devices. So cybercriminals leveraging that. For you, the user, nothing looks unusual.

There’s no Comic Sans, there’s no Nigerian prince promising you anything. Instead, they’re putting more effort in than most phishing emails.

So the email they send you is an actual Microsoft device login page.

Now, I don’t know about you, but as an industry for 20 years, we’ve been telling people, you know, always check that you’re actually on a real website.

You know, check the URL, make sure it looks real and genuine. And congratulations, in this case, you are.

So you enter the code, and the code is generated by the attacker’s Greatness toolkit.

Unfortunately, that application actually belongs to someone whose hobbies may or may not include cybercrime.

And you naturally go, yeah, ’cause you think you’re approving your own login. Instead, you’ve just approved theirs.

And so this is what’s been described as the sort of cyber equivalent of someone knocking on your front door and said, would you mind helping me carry your television out to my van?

And you’re going, oh yeah, of course. Safety first.

So the Microsoft page, which is the real Microsoft page, the real deal, is asking you for your permission to connect someone else’s dodgy bit of code or whatever it’s going to do with your account so it can read and steal all of your information.

And again, because it’s OAuth, they don’t need your password. You’ve just essentially handed them a VIP pass into your accounts.

In terms of, if you’re thinking from an enterprise piece, their advice is to disable things like device code flow if you don’t use it, because a lot of conference televisions are just screens to display information, right?

And then the other thing is to restrict OAuth applications to only users that consent to use it. So getting some role-based access control in there.

If you can, put some extra monitoring in place for OAuth to see about consent events, see if there’s anything unusual going there.

And I guess the other thing, which kind of goes against what we’ve always discussed, is maybe we need to start teaching users that trusting a website really isn’t enough anymore.

I mean, presumably they could have a library or a database of approved third-party apps, which they’ve sort of said, you know, these are legit ones you may want to connect to your account.

And this one has only been created maybe 3 days ago and therefore warning, warning, warning, warning. You may want to think twice before approving this.

I mean, maybe Microsoft could display something just to make people stop and think a little bit like, you know, unless you have actually asked to authorise this, then stop right now.

So yes, we may see something like that come in the future, but at the moment, it’s a free-for-all. And so maybe greatness really is great at doing this.

But I guess the only advice I’ve got for you is make sure it’s yourself that’s logging in and not some bloke from organised crime.

I’ve not heard many of the other bigger news stories picking this up. And it’s terrible.

That’s V-A-N-T-A.com/smashing. And listeners, you can get $1,000 off.

Pick of the Week.

It doesn’t have to be security-related necessarily.

So Lianne, you are old school.

As nerds, and if you’re listening to this, you’re probably more technical than the average person in the street, and you probably get lumbered from time to time helping out friends and family remotely with their computer problems.

Your Auntie Ethel, her computer has broken down and she can’t get it to work, or it’s bizarre error messages. She doesn’t know what to do.

You don’t wanna have to drive around there trying to fix it.

Well, you can try pairing Tailscale, which is a dead simple VPN that connects your devices securely, with RustDesk, which is an open-source remote desktop tool that works without a middleman.

Now, both of these are free to get started. RustDesk is completely and utterly free. Tailscale has a very decent and acceptable free plan as well.

And together they can give you remote access that you can actually trust. By the way, although I said Tailscale is a VPN, it’s not a traditional VPN.

It won’t pretend that you’re in a different country.

It’s designed to connect your devices to each other securely, so it’s not about hiding your location, although it has an exit node feature which can route your traffic through a computer at your home.

But fundamentally what it will do is connect to your other computers. So it’ll connect to your computer running at home or your Auntie Ethel’s computer at her place.

So rather than exposing my machine to the open internet or trusting a third-party service with my connection, I don’t like to use TeamViewer and things like that where you’re going through someone else’s servers.

Tailscale creates a private encrypted tunnel between your devices and RustDesk, if you’re running that as well, never touches the public internet at all.

It stays entirely within that private network. So you’re running RustDesk inside Tailscale, if that makes sense. So you can do anything you want.

So I can just take a little laptop or an iPad with me and I can connect to my home proper computer and do things if I need to do them.

All acts with magic, not going on any third-party servers. You can even access your computer from your mobile phone.

You’re spreading your hands a bit like a man would on a dating app when he’s demonstrating the size of fish which he has caught.

And then we ask each other the following questions. So I’m gonna ask, I’ve got some example question cards here.

You have a full stop. You have—

So apparently the Greeks adapted the Phoenician alphabet sometime between 1000 BC and 850 BC, adding vowels to what had previously been vowel-free. They are the first to do this.

And so it not only gives you the answer, it tells you all about it. So it’s a very intellectual game. You’ll be smarter for playing it.

Is it confer? Is it confirmo, confuto, or confirmato? Which means compare, confirm, repress, or shake. What does CF, the abbreviation C.F.?

It’s actually compare.

It is commonly seen in reference to text and indicates a term that should be compared with something else, as in Perinthian shop or parting shot.

You can get it at Waterstones. How much does this cost?

I’m sure lots of our listeners would love to find out what you’re up to and follow you online. What’s the best way to do that?

I can’t be held responsible if you just type in Compromising Positions. And also the new Tech Film Noir.

And you can find me up on those places as well, as well as on LinkedIn. And don’t forget to ensure that you never miss another episode.

Follow Smashing Security in your favourite podcast app, such as Apple Podcasts, Spotify, and Pocket Casts.

Episode show notes, sponsorship info, guest list, and the entire back catalog of 480 episodes. Check out smashingsecurity.com. Until next time, cheerio. Bye-bye.

And huge thanks, of course, to Lianne Potter for joining us this week and to this episode’s sponsors, Arctic Wolf, NordLayer, and Vanta.

And we’ve also got to thank our fantastic Patreon supporters, members of Smashing Security Plus. We’re going to pull some out of the hat right now. Julian Beach, tremendous name.

Sounds like somewhere you’d go for a very nice holiday. Adina Bogut O’Brien, she’s back again with the hyphen and the apostrophe. Very impressive.

Still probably the most punctuated name on the list. Vladimir Jirasek, who sounds like, well, he— a lot of final boss energy about him, I suspect.

Mark Norman, Andrew Davison, Skadone, still unflinchingly lowercase Skadone, still not explaining anything about that name.

Panos, Tepotastic, which I don’t know if that’s a Finnish name or a very, very enthusiastic adjective. Maybe it’s both.

Heisenberg, they may or may not actually be listening right now, really is hard to tell. Matt H. The H is still classified mystery, still very much alive. Thank you all so much.

You are absolutely wonderful.

Those are just a few members of Smashing Security Plus, which means that they get their episodes ad-free and earlier than the general public, and they can have their names drawn out of the hat at random to be mercilessly mocked at the end of the show.

If you fancy a bit of that, just go over to smashingsecurity.com/plus for all of the details. It’ll cost you probably less than a cup of fancy coffee, I suspect.

Not being a coffee drinker myself, I’m not really sure what I’m talking about. But anyway, I imagine it costs about that much.

You can become a patron, but you can also support the show in plenty of other ways which don’t cost a penny.

You can like, you can subscribe, you can leave a five-star review wherever you listen, and you can of course tell your friends about the show.

I think that’s one of the most effective ways you can help, and I really do appreciate it. Spread the word because every little bit helps, and it makes all of the effort worthwhile.

Well, I hope you will tune in again next week for another episode of Smashing Security. Cheerio! Bye bye.