Smashing Security podcast #465: This developer wanted to cheat at Roblox. It cost millions

A developer at an AI startup wanted to cheat at Roblox. They downloaded a dodgy script on their work laptop. That one decision triggered a cascade of failures that ended with a $2 million data breach affecting hundreds of thousands of organisations. All for some free in-game currency.
Meanwhile, there’s a 1980s phone protocol called SS7 that lets shadowy surveillance companies track anyone, anywhere, via their mobile phone. Governments know about it. Telecoms know about it. Nobody’s fixing it.
All this and more in episode 465 of the “Smashing Security” podcast with cybersecurity keynote speaker and industry veteran Graham Cluley, joined this week by special guest James Ball.
Plus! Don’t miss our featured interview with Rob Edmondson of CoreView, discussing how to lock down Microsoft 365 before it’s too late.
0:00
0:00
Show full transcript
▼
This transcript was generated automatically, probably contains mistakes, and has not been manually verified.
This is like 4 layers of Swiss cheese lining up and just something dropping straight through, isn’t it?
It’s not staying contained at the moment. And so I’d like to put it on the record, I am the greatest victim of the world’s political situation right now.
You know, I had some feedback from a listener just in the last couple of days actually saying, you love the podcast, been listening to the podcast forever, but oh my God, Graham, can you stop talking about politics?
And my reaction was, look, thank you very much for listening and all the rest of it, but it feels to me that technology and politics are more intertwined than ever before.
You can’t really extract them from each other, can you?
I’ve sort of always gone, I wish people would pay more attention to this. This is transformative. I sort of came of age with the internet.
You’re kind of going, no, we need to look at this. This is really huge.
And now tech and politics have merged so much and are in the discourse so much and are crashing together in so many ways. It’s I was really stupid to want this.
Why can’t this go back to being in its nice own lovely corner where I can just think about how the technology works or, you know, the principles of it instead of what stupid way is this going to be used to upend our politics yet again?
How can you extricate them? I don’t know that you can.
It’s when the railway monopolies were there or the early oil monopolies, because the biggest companies and the biggest tech companies is synonymous.
9 of the world’s 10 biggest listed companies are tech companies. Essentially, this domination by one sector is pretty much unheard of in either of our lifetimes.
And so politics is going to be weird until tech is kind of normal again. And that might be bad for someone who reports on and covers tech, but might be good for the world.
It might be good for our blood pressure and it might be good for your listener. I promise I haven’t brought a load of political things this week.
Well, not very political, small p political.
Well, before we kick off, let’s thank this week’s wonderful sponsors, CoreView, Elastic, and Vanta. We’ll be hearing more about them later on in the podcast.
This week on Smashing Security, we won’t be talking about how home security firm ADT has been burgled by the Shiny Hunters gang.
You’ll hear no discussion of how ransomware negotiator has pleaded guilty to helping hackers by leaking victims’ insurance details.
And we won’t even mention how Elon Musk’s Grok chatbot told researchers pretending to be delusional that there was indeed a doppelganger in their mirror and they should drive an iron nail through the glass while reciting a psalm backwards.
So James, what are you going to be talking about this week?
He’ll be joining us as we take a look at how hackers have been turning essential tools like Microsoft 365 against their targets and what you can do to lock down your environments before it’s too late.
All this and much more coming up on this episode of Smashing Security.
If someone broke into your Microsoft 365 tenant right now and quietly disabled your conditional access policies, grabbed global admin rights, turned off Bitdefender, would you even notice?
One compromised account and an attacker can quietly reshape your entire tenant.
No alerts, no noise, just someone systematically dismantling your defenses while you’re none the wiser.
You could be rebuilding your tenant settings from scratch for weeks.
It’s actually a really practical read.
It covers how these attacks unfold step by step, where your existing tools are leaving gaps, and what it actually takes to recover control once it’s been lost.
You can learn more at smashingsecurity.com/coreview and maybe do it before someone else does something bad to your organization.
They are at the heart of all kinds of stuff which is going on on the internet. Hundreds of thousands of organizations use them because they’re a cloud company.
You know, they’re a properly grown-up company. And on the 19th of April, Vercel put out a security bulletin. I’ll summarize it.
They basically said, we’ve been hacked, customer data has gone missing.
We’d quite like to tell you about it before the Russian hackers selling it on a darkweb forum get there before us. So we’re going to get out there ahead of the bad news.
Now, normally at this point, I’ll be telling you about a clever zero-day vulnerability or sophisticated nation-state campaign or even a simple phishing email, right?
Normally, that’s the kind of thing which I’m— not this week, however. No, this week, James, the story really begins with someone wanting to play Roblox.
I am less terrible at Roblox than I am at Minecraft. I think really it’s not for adults, it’s for teenagers. It’s kind of even for tweens, really.
I’m sure there are adults who enjoy Roblox completely legitimately, but it has been slower to act on those concerns than almost any other platform as well.
I hear they have improved of late.
They’re not the Context AI which was acquired by OpenAI.
It’s a different Context AI, which I think really suggests that, you know, maybe people shouldn’t have relied upon AI to dream up their company name.
Clearly they were using an old version of a model there because that is quite important context, ironically.
So it’s something which plugs into your Google Workspace and you grant it sweeping permissions and it can go ahead and read your email and your documents, helpfully does all kinds of wonderful AI things to them.
It sounds absolutely gorgeous. You know the kind of thing, people get it all the time. A consent screen will pop up.
You know, a sensible person reading the terms and conditions go, ha ha ha ha ha, no way. Eh, eh, it’s a big fat no.
But many people will just hit the approve button instead in order to allow it to do that.
And apparently one of Context AI’s own employees, someone who works at the company, a company which has asked its customers to trust them with the keys to their corporate Google accounts, effectively, they, on apparently a work laptop, decided what they really needed in their life was a Roblox auto-farming script.
Are you familiar with auto?
I mean, just the fact anyone working in any tech-related sector would do this. Gaming add-ons are notorious. You know, approved add-ons, fine, great.
I’ve modded almost every game I’ve ever played. That’s the fun of it.
You know, Baldur’s Gate 3, absolute delight and amazing mods on that and some slightly horrifying ones involving Withers that I won’t get into. If you’ve played the game, you know.
But any mod that sort of claims to let you do something a bit against the rules tends to be incredibly dodgy because they never go through the official stores, which means—
Or, you know, trustworthy.exe from a Nigerian prince, Derby Dragons stuff.
But you’ll get the in-game currency so you can then buy things, you know, add-ons and so forth. And it’s dodgy software, as you said, James, downloaded from dodgy websites.
But people have already decided the rules don’t apply to them and they’re encouraging it.
I mean, it always sounds daft getting something to play the game for you, but any sort of massively multiplayer game has bits that basically involve grinding.
You know, it’s a bit if you could pay someone to go to the gym for you and you got the results, which, you know, the dream.
I mean, look, no company IT server setup should ever have let someone be able to install this, really, should they? I mean, this is a bit of a disaster.
You know, the user is stupid here, let us stress, but they shouldn’t have been able to make this mistake, should they?
And unfortunately, this particular script came bundled with the Lumma, I believe it’s pronounced, Info Stealer, which rifles through your browser, grabs your passwords, every cookie, every session token, every OAuth credential it can find.
Bundles it up, sends it to a complete stranger afterwards. So in February, this guy downloaded this Roblox cheat. He got infected.
Lumma quietly stealthed its way into the browser, grabbed the database of information, including Google Workspace credentials, including the keys to Context AI’s AWS environment, including— and this is the crucial part, really, I suppose— including the OAuth tokens belonging to Context AI’s customers.
Well, this certainly shouldn’t have been able to happen. Oh, oh, this is great.
I mean, this is sort of building an incredibly elaborate safe door with all of this sort of stuff and then just leaving the code on a Post-it on it, isn’t it?
Graham, this is not great.
So when you click allow, you are giving an app, for instance, access to your Google account. They don’t need your password. They don’t trigger your two-factor authentication.
And once a thief has your OAuth token, they don’t need to break in because as far as Google’s concerned, they are you.
And the scary thing, I think, for many people, and they don’t realize this, is if they actually check their Google account right now and have a look at what apps they have granted access to their account over the years, they’re probably going to be surprised.
There’s probably things in there that you don’t remember doing, or you may have just done on one particular day, and you’ve granted them access to stuff, and you should revoke it.
You know, I think there were several on Twitter, as it was back in the day, that would give you a score for your social standing or your clout.
But these old apps that you’d granted access to for one purpose would get bought by someone else or the domain would get taken over and they could hijack the thing.
And so even things that were completely sensible to grant access to, suddenly became terrible.
I remember back in the days of when I was on Twitter, as was, there was a third-party app or something or service which I think was doing some kind of ego-stroking examination of my followers, right?
So I could think I was doing really, really well in terms of Twitter followers.
And what happened was that particular service got hacked which means the hackers then had access to my Twitter account, not just me, but also Justin Bieber and whoever else.
And my account started posting Nazi spam to people. And you just think, oh no, no, no, I don’t, you know, I don’t want this. So it can happen to everyone.
You always need to look and revoke permissions wherever possible.
So one of Vercel’s employees had at some point signed up for the Context AI office suite using their Vercel Enterprise Google Workspace account.
And when the permission screen came up, they clicked on Allow All.
So now our hacker, our attacker, who started his day off poisoning Roblox hacks, is sitting on an OAuth token that gives him read access to a Vercel employee’s entire Enterprise Google Workspace.
So you’ve got different companies here, but it has cascaded through to grant a huge amount of permission to access data.
This is like 4 layers of Swiss cheese lining up and just something dropping straight through, isn’t it? Have you not come across this analogy? I didn’t just make it up. I haven’t.
And so you just get more and more of them on top of each other and you reduce the chance of them lining up. This is like a hole through 5 slices, just straight there. Bumpf.
Now, Vercel says that some of these were marked as sensitive and therefore protected, but the ones which weren’t marked as sensitive, which apparently were most of them, because that wasn’t the default, duh, once again, they’ve changed that default now, by the way, funny that.
Yeah. Regardless, this data is now listed for sale for $2 million, all because someone at Context AI wanted to cheat at Roblox, downloaded malware, the malware stole the tokens.
Those tokens belonged to a Vercel employee who had secured his unsanctioned AI tool to their corporate Google account and clicked allow all.
I doubt he’s managed to create himself $2 million though to go and buy the data for himself to prevent it falling into the hands of anybody else.
So your team logs into tool 1 and then maybe tool 2, then into the thing that doesn’t quite talk to either of them. By which point, whatever was happening has—
And CitizenLab, they’re a sort of Canadian-based, not-for-profit research grouping, and they do some really impressive work on security and surveillance.
And they’ve had quite a long interest in people who expose phone networks.
So this isn’t the News of the World hacking your phone, but this also isn’t some of the Black Cube or the Israeli-type security companies hacking your individual handset so much.
This is about people using the actual architecture of phone networks to track your location, to track your SIM card, sometimes to try and put tools on your device.
And one of the key ways they’re doing this is basically either posing as a mobile phone company and getting access to the towers that way, or working with some unscrupulous mobile phone companies to sort of get in.
And this is an absolute bugbear to— so for most people, this is not, you know, your mobile phone location data isn’t super interesting.
And they managed to get her phone geolocated and raided the boat she was on and recaptured her. And she’s basically never been seen in public since.
People sort of look at political enemies and exiles overseas. It is quite bad. And there’s always talk about regulating the companies that do it.
But the reason it irritates me so much, and the reason that I brought it, is because I commissioned an investigation on this years ago when I worked at the Bureau of Investigative Journalism.
There’s a guy called Crofton Black who has been doing reporting on this for more than a decade now. Look up his work, look up Citizen Lab. They are the experts here.
And I should stress, I am not an expert on mobile phone infrastructure, so please take mine as a hopefully roughly correct explanation of this.
And if people want the proper stuff, look at the Citizen Lab report because they are much better than me.
But when that’s not available or you have low signal, your phone reverts to 2G or 3G.
And you may have found that sometimes if you’re out in the sticks or frankly sometimes in central London, you don’t get data, but you can make a phone call or you can get a text.
It basically assumes pretty much anyone who’s got a tower, who’s in that backend network is trusted. And it will let people hand over metadata, let people take location information.
It will let them query this stuff and pull it through with absolutely minimal security.
This protocol is, I think, about 30 years old, and it was set up for when phone networks were quite basic, when they were quite early.
This is the classic story of internet protocols not being secure by design. Look at Border Gateway Protocol, look at until fairly recently, DNS, look at whatever you like.
It is very much from that tradition.
But the issue is because global rollout of 4G and 5G has been quite slow, because people need fallbacks for things like emergency access, emergency numbers for disaster recovery efforts, SS7 is still built into almost everything.
It’s only now being deprecated anywhere. And so at a protocol level, mobile phones are just fundamentally insecure.
And everyone in this very niche world has known about this for at least 10 years.
There’s no political attention on it. There’s very little media attention on it. It’s quite complicated. And it kind of only affects particular people who would be targeted.
But it puts all of us at risk. It is incredibly dumb. And I do not know why it’s been tolerated for so long.
And it’s because everyone likes having that 2G, 3G layer to fall back on. Because it’s not used very much, it’s reliable, it’s basic, it’s dependable.
It can work on much lower levels of signal than some of the modern ones. It’s been quite useful to have it sitting there for things like reliability, for emergency, for fallback.
And rather than replace or fix it or go, well, why don’t we use this spectrum but with a better protocol? People have just gone, well, obviously, you know, we can’t fix SS7.
Why pay attention to that? Why don’t we look at 6G? Why don’t we look at, you know, 5G+ et cetera?
And so everyone acknowledges it should be fixed, but I don’t think anyone thinks it’s their job to fix it. Or this has been my impression.
There’s been very little political or regulatory pressure on it. There have been a lot of actors who want to exploit and use it.
And it’s not been the top of anyone’s agenda because ministers want to say, “I’m going to get you ultra-fast, you know, new mobile broadband, and that will boost GDP.” No one wants to go, “Hey, that creaking old bit of the phone network that no one’s heard of, I fixed that.” No, not so sexy, is it?
I mean, you’ve already mentioned this sort of Jersey way of breaking into +44. Which is the UK’s country code.
I think people listening to this have probably heard of Stingrays.
That’s the kind of technical way of pretending to be a phone mast. This is like a business way of pretending. You sort of say, hey, I’m a new virtual mobile phone provider.
I’m a new virtual network. I’m going to have phone customers.
And you sort of team up with a real phone network to get their infrastructure, and they have to data share so that you could work a network.
And then they actually use it to make these inquiries that they shouldn’t be doing.
And I think one of the issues that came up was very few people actually log how people were using these queries and whether they were restricting them only to their own customers, et cetera, because it never occurred to them to put it in, you know, and I think it was possible to audit this for various reasons.
You just can’t fix SS7 as a protocol.
It’s not like you could just do a patch, but this you could have done and just gone, well, if anyone’s querying more than X times a day relative to their customer numbers, you know, we cut them off or we investigate.
I think you could have put quite a lot of exceptions in. And I think that’s what the bigger companies do. I think that’s why you need these little backdoors.
But for as long as there are little jurisdictions that can get you into bigger ones, as long as there are smaller phone companies that don’t care very much, this will remain very vulnerable.
And this is quite bad. There is a roaring trade in exploiting this. You know, there are security companies getting quite rich off this.
You know, the fact that there is a roaring trade in this tells you that there is a vulnerability and tells you that there is a problem here.
If this was just academic, companies wouldn’t be trading off it in this way, you know?
And as I say, the getting snatched off a boat while trying to escape your dictatorial father is the extreme limit.
You know, we know the ways that this stuff can be exploited and it can be absolutely brutal. And so it’s fairly unforgivable that it’s not been fixed in so long.
Sorry, this is a lot less fun than your topic.
I mean, it feels like there’s quite a contrast here between this and spyware like Pegasus, which impacts individual phones, because this is surveillance you’re talking about happening at a network level, meaning even a perfectly secure handset can be tracked, one that’s properly locked down.
I mean, especially, I don’t know where you quite land on Mythos, but I sort of land to thinking ultimately it’s going to be a defensive advantage because we’ve known that everything’s had zero days in it since forever.
I think you actually got to get a defensive equilibrium. I think things like Pegasus are going to suffer in the Mythos era.
And so it’s going to become more urgent to address these things. And that’s a shame because we could have done this at any point in the last 10, 15 years.
Should they actually care about this? Is this really only a problem if you’re a journalist or a dissident or a high-profile target? What’s your feeling on that?
There are some people who have had some very unpleasant emails or some odd things because they have the same name as me.
And I have at various times been tracked and surveilled by various governments for my sins.
And if you happen to be called James Ball and you’ve done nothing wrong in your life and suddenly you’re exploited because of this, that’s not great.
So mistaken identity can get you.
A pal of mine, Hassan Akkad, was a schoolteacher in Syria, and he taught English in schools until the civil war started, and he started seeing people get disappeared and tortured.
And he started filming that, and he was then himself detained and tortured and managed to escape. He’s a British citizen these days. I’ve even seen him streak at cricket.
How’s that for naturalisation?
He had a very normal middle-class life in Syria until he suddenly didn’t. And we sort of see how the world is changing and all these things are happening.
And yeah, this stuff is all very far away from you until very suddenly it isn’t.
And so look, for most people listening to this, the thing you should worry about is actually, have you done the software update?
On your phone, because if your apps are up to date and your software is up to date, that is way better than anything else you can do.
It automates all of that tedious manual compliance work so you can stop drowning in spreadsheets, chasing audit evidence, and filling out questionnaire after questionnaire.
It also uses AI to streamline evidence collection and flag risks. It automates compliance for SOC 2, ISO 27001, HIPAA, GDPR, and more.
Could be a funny story, a book that they’ve read, a TV show, a movie, a record, a podcast, a website, or an app. Whatever they wish.
It doesn’t have to be security related necessarily. Now, James, do you remember a TV show called Name That Tune?
And they’d have to bet in how few notes, you know, plonked out on the piano, they would be able to name that tune in.
And someone goes, donk, and they’d say it’s whatever it is, you know. Now, James, I know that you love people sharing their musical tastes on social media.
And so one year I did why I hate It’s a Wonderful Life.
And I’m sorry, your wife becoming a librarian and having a career is not a fate worse than death. But that was one year.
And I think this was last Christmas I did, “Your Spotify rap is not interesting, so please shut up about it.” So yeah, every Christmas I try and ruin something.
Now, listeners know I’m a bit of a fan of the Beatles, and they released 213 tracks during their active career during those seven years.
And hence, I have been having a lot of fun playing a game online called Think for Yourself at thinkforyourself.live.
And I thought, James, I don’t know if you know any Beatles tunes at all.
Okay, and I’m going to play a second and see if I can name the songs just to demonstrate. And hopefully also won’t be copyright infringing.
