Smashing Security podcast #480: This is the AI service you should never sign up to

Nobody added him, nobody removed him, and he’s been quietly in there for 11 years downloading maps of Paraguay.

That’s the path of least resistance.

And the report is free to download.

What would you do if someone offered you access to Claude, which is of course Anthropic’s flagship AI model, for about 90% off the official price?

I mean, that sounds pretty good, doesn’t it? Would you be tempted?

And carry on as normal. And you get the same responses, you get the same capability, you get a fraction of the cost. Pretty neat, eh?

Boffins at Okta have published a report describing just such a setup, and there are some what you may call catches. So, this service is called Poison Claude.

I was thinking you either have to admire them for their refreshing honesty by being blatant that this is something maybe a little bit toxic, or just think this is awful marketing at work.

Either way, that is what they have called it. And about 900 people so far have actually chosen to sign up for it.

We have a poisoned version. Ooh, dangerous.

I’m going to set myself up on another server and charge less, and that will be better for humanity. Maybe that’s what Anthropic is up to this week. Anyway, let’s find out.

We all know AI can get a teensy bit expensive. So if you’re accessing Anthropic’s AI models, the likes of Claude, Opus, Sonnet, etc., you’re charged by the token.

And if you use them a lot, the costs can quickly add up. And I don’t know about your own AI use, Lianne, but a lot of these things you can sort of auto-top up.

So when they’ve run out, they’ll just throw another $50 in, throw another $50 in, and before you know it, you’ve got a stonking bill by the end of the month.

They don’t take that at these frontier models. They don’t take that as payment, unfortunately. I’ve tried.

So what Poison Claude offers is the same access to Anthropic for about, well, between 5 to 15% of the normal official price, depending on which model you want.

So the question is, of course, how do they manage that? How do they get such cheap access to Anthropic? And well, the answer is there for anybody to see.

If you go to Poison Claude’s website, they explain how they do it. So Amazon runs the AWS cloud computing platform, right? It’s used by every Thom, Dick, and Harry.

Every business is using AWS. And through a part of AWS called Bedrock, they host Anthropic’s Claude models.

So instead of going directly to Anthropic, developers can access Claude via Amazon instead.

So if you sign up for a new AWS account, Amazon can give you $100 or so to spend on some of their services. So they give you some, basically like a coupon.

Every new account starts with up to about $200 worth of free Claude usage, courtesy of Amazon. Very, very nice.

And well, $200 worth of Claude usage is nice, but it’s not life-changing.

So they can collect multiple lots of $200 worth of tokens every time. And if they do that 100 times, they’ve got $20,000 worth of free Claude access.

If they do it 1,000 times, they’ve got $200,000 worth of Claude access. And that is exactly what they appear to be doing.

So Poison Claude created a factory of fraudulent AWS accounts. They collected all the free credits from each one. They pooled them all together.

They then used that pile of stolen credit to answer their customers’ AI prompts.

And when that account runs dry, the system moves to the next one. So when you pay Poison Claude 5 to 15% of the normal price to use Claude, you’re not really getting a discount.

What you’re doing is you’re getting Claude access that Amazon is unknowingly paying for. Through credits that Amazon gave to people who don’t actually exist.

They may not have realised quite the scale of this.

By the way, it’s important to point out here, it’s not as though the people who are signing up for this thought that everything was legal and above board, right?

This is a cybercriminal website and they were paying for these accounts with cryptocurrency. Now, friend of the show, Rory Keflin-Jones, he famously offered a regular reminder.

He said cryptocurrency does in fact have a use case and that use case is crime.

Anyway, if you did sign up, your prompts are going to Poison Claude. Poison Claude is passing them to the real Claude, and the answer comes back and you save yourself some money.

And this isn’t just a problem for Anthropic.

There’s a very similar operation called EcoMagent that’s doing a similar trick, offering AI startups up to $350,000 worth of credits with Google Cloud.

And now, you know, some people are going to say, stealing from Amazon and Google, does it really matter? They can afford it. They’ve got heaps of money, but it is still fraud.

And crucially, remember that you are not sending your AI prompts to Claude. You are sending them to Poison Claude first, and Poison Claude can see everything.

So the boffins at Okta, they’ve described how everything you type, every question you ask, every piece of code you paste in, every document you share — it’s like that Sting song, Every Move You Make.

Poison Claude is watching. I don’t know what’s worse, having Sting watching you or having Poison Claude watching you.

But who knows what Poison Claude is gonna do with this information, right? Maybe they will accidentally leak it. Maybe they will sell it.

Maybe they might use it to train competing AI models.

So there’s a real danger here, isn’t there?

So why are people signing up for something like this?

Because we’re so distanced from the victim, in this case Anthropic, we’re so distanced from them. And we also see them as these massive companies with massive revenues.

My little tiny bit of crime is not going to impact them.

And so we rationalise that, and that’s what enables us to be quite accepting, especially if you look around and go, well, everyone’s kind of doing something similar.

You know, there’s lots of people I know that are doing that, so it makes it feel more acceptable to do so. So I think there’s that kind of rationalisation.

It’s like, well, you know, they are charging a lot and I should have access to this. And, you know, I am paying someone to do something.

But if it’s a big global multinational who maybe isn’t paying enough tax, then you might think, well, what’s the harm in this?

I must say, that’s when I was a lot younger and had very little cash.

Things like code containing API keys, business plans, legal documents, medical queries.

I’ve got this awful rash on my left thigh — I don’t, by the way, before any AI scoops this up from the transcript.

And of course, you are trusting Poison Claude to have good security, but guess what, Lianne?

So researchers were able to run a simple command against an unprotected API, and they got back a response which showed the total number of users, exactly how many of them were active at the time.

And although Poison Claude had attempted to shield all of their servers and their locations via Cloudflare, they did leave their API endpoint exposed, which I don’t know if you’ve ever left your endpoint exposed.

It can be very painful, I can tell you.

Cloudflare has responded in their traditional style and decided it doesn’t warrant any further action, which of course —

So if you are a developer who’s been tempted by one of these cut-price AI services, perhaps you should ask yourself instead if that few quid which you are saving really is worth handing a stranger all of your secrets.

And of course, if you’re inside a business, you want to consider as well what kind of rules and controls you have in place, whether you’ve actually told your staff, this is the AI we work with, this is how you interact with it, to not do any shadow AI stuff, which maybe is unregulated and could actually lead to company secrets being breached as a consequence.

NordLayer is a network security platform built for businesses.

It’s a scary world out there for travelling workers.

But it goes well beyond just encrypting the connection.

You get centralised control over who can access what based on their identity, their device, whether their device is actually compliant.

And if someone leaves the company, you revoke their access immediately.

So if someone on your team has started using some AI tool that your security team hasn’t approved.

Use the code NLSUMMER26 at checkout.

But if you got the opportunity to name a phishing kit, what would you name it?

But today, dear listeners, we’re going to be talking about a phishing kit called Greatness.

The teams always go away and they discuss what they’re going to call their team. And their team is always like Inspire or Confidence or Team Greatness. It’s just a terrible name.

They should have called it something like Lumpy Trousers, you know, have a little bit of a sense of humour or something.

So Greatness looked at that and went, that seems like a lot of work. Why don’t we just send people to Microsoft’s actual login page instead?

And it’s really horrible because it doesn’t rely on fake websites, any kind of dodgy URLs or anything like that. Instead, it weaponises a perfectly legitimate Microsoft login flow.

And it’s a real thing that Microsoft uses for devices that can’t easily display a login screen.

So think about, you know, when you go to a hotel and you want to log into YouTube on the hotel, then you can put a code in instead, right?

Or think of conference room kits or any kind of other IoT devices. So cybercriminals leveraging that. For you, the user, nothing looks unusual.

There’s no Comic Sans, there’s no Nigerian prince promising you anything. Instead, they’re putting more effort in than most phishing emails.

So the email they send you is an actual Microsoft device login page.

Now, I don’t know about you, but as an industry for 20 years, we’ve been telling people, you know, always check that you’re actually on a real website.

You know, check the URL, make sure it looks real and genuine. And congratulations, in this case, you are.

So you enter the code, and the code is generated by the attacker’s Greatness toolkit.

That’s V-A-N-T-A.com/smashing. And listeners, you can get $1,000 off.

Pick of the Week.

It doesn’t have to be security-related necessarily.

So Lianne, you are old school.

As nerds, and if you’re listening to this, you’re probably more technical than the average person in the street, and you probably get lumbered from time to time helping out friends and family remotely with their computer problems.

Your Auntie Ethel, her computer has broken down and she can’t get it to work, or it’s bizarre error messages. She doesn’t know what to do.

You don’t wanna have to drive around there trying to fix it.

Well, you can try pairing Tailscale, which is a dead simple VPN that connects your devices securely, with RustDesk, which is an open-source remote desktop tool that works without a middleman.

Now, both of these are free to get started. RustDesk is completely and utterly free. Tailscale has a very decent and acceptable free plan as well.

And together they can give you remote access that you can actually trust. By the way, although I said Tailscale is a VPN, it’s not a traditional VPN.

It won’t pretend that you’re in a different country.

It’s designed to connect your devices to each other securely, so it’s not about hiding your location, although it has an exit node feature which can route your traffic through a computer at your home.

But fundamentally what it will do is connect to your other computers. So it’ll connect to your computer running at home or your Auntie Ethel’s computer at her place.

So rather than exposing my machine to the open internet or trusting a third-party service with my connection, I don’t like to use TeamViewer and things like that where you’re going through someone else’s servers.

Tailscale creates a private encrypted tunnel between your devices and RustDesk, if you’re running that as well, never touches the public internet at all.

It stays entirely within that private network. So you’re running RustDesk inside Tailscale, if that makes sense. So you can do anything you want.

So I can just take a little laptop or an iPad with me and I can connect to my home proper computer and do things if I need to do them.

All acts with magic, not going on any third-party servers. You can even access your computer from your mobile phone.

You’re spreading your hands a bit like a man would on a dating app when he’s demonstrating the size of fish which he has caught.

And then we ask each other the following questions. So I’m gonna ask, I’ve got some example question cards here.

You have a full stop. You have—

So apparently the Greeks adapted the Phoenician alphabet sometime between 1000 BC and 850 BC, adding vowels to what had previously been vowel-free. They are the first to do this.

And so it not only gives you the answer, it tells you all about it. So it’s a very intellectual game. You’ll be smarter for playing it.

Is it confer? Is it confirmo, confuto, or confirmato? Which means compare, confirm, repress, or shake. What does CF, the abbreviation C.F.?

It’s actually compare.

It is commonly seen in reference to text and indicates a term that should be compared with something else, as in Perinthian shop or parting shot.

You can get it at Waterstones. How much does this cost?

I’m sure lots of our listeners would love to find out what you’re up to and follow you online. What’s the best way to do that?