UnHacked - Cybersecurity Made Simple for Small Businesses

Justin Shelley (00:04)
Dancing going on.

It's always a a crapshoot to see who's gonna dance and who's not gonna dance to the music. Today I got a little bit in there.

Mario Zaki (00:16)
I I don't even know if that worked.

Justin Shelley (00:19)
but we we we lost Craig. Craig, welcome back to the show. We I don't remember what your episode number. I should have had that handy. But you joined us what a couple months ago, a few months ago? Do you do you remember how long ago? Okay. Do you remember your episode number? No,

Craig Taylor (00:31)
Yep, a couple months ago, it was at least two months. No.

Justin Shelley (00:36)
of course. anyways. Are are you sure? Are you making that up?

Joshua Holloway (00:38)
Sixty three.

No, I'm making it up.

Justin Shelley (00:43)
God damn

Craig Taylor (00:43)
If I

was to make something up, I'd pick a different number, but I won't.

Justin Shelley (00:47)
Okay. So Craig

Joshua Holloway (00:48)
Add six.

Justin Shelley (00:50)
Listen, rein it in right now or I'm gonna shut off your mics. Craig the CEO of is that your proper title, Craig CEO? Okay, Craig Taylor, CEO of

Craig Taylor (00:59)
Yes, CEO and Chief

Fish Whisperer.

Justin Shelley (01:03)
Chief Fish Whisperer co-founder or just founder? You've got another

Craig Taylor (01:06)
co-founder of Cyber.

Justin Shelley (01:07)
co-founder of Cyberhoot security awareness training. Yeah. Great

Craig Taylor (01:13)
Mm-hmm. Yes.

Justin Shelley (01:14)
stuff. I've I keep promising you that I'm going to integrate it into my platform, the unhackmybusiness.com, that I keep talking about. but I'm

Craig Taylor (01:21)
You will.

Justin Shelley (01:22)
a liar. I just keep talking about it and not doing it. It'll happen eventually. But I've gotten so into

Joshua Holloway (01:26)
Scope creep is getting ya.

Justin Shelley (01:28)
scope creep is the phrase of the day. Once I started vibe coding,

All my little inner child stuff came out. I I listen, I had my first computer was an Apple IIe back in the eighties. and and writing code was my love. Hell yeah. Under the monitor.

Craig Taylor (01:40)
Did you get the dual disk drive? Did you have, that was awesome. you, and

under the monitor, the, was about this thick, two five and a quarter inch floppies, not the little ones, the five and

Justin Shelley (01:49)
Yep. No.

Craig Taylor (01:51)
a quarters. And you could punch, hole punch them and flip them. And you could have double the storage. Who remembers that?

Justin Shelley (01:54)
Yeah. yeah. yeah. I do.

And and the three and a half inch floppies that weren't floppy, right? Because they had a hard case on them, but the disc inside was I

Mario Zaki (02:02)
Yeah.

Craig Taylor (02:03)
No. Yeah.

Justin Shelley (02:04)
don't know. Anyways, yeah, so coding really is my my love and my passion. And it it since I

Craig Taylor (02:13)
But you're gonna be out

of a job soon, because AI is gonna do all that for you, isn't it?

Justin Shelley (02:17)
No,

because now I don't have to. I just tell it and it it does all the coding for me, but I still get the same fix. The fix wasn't writing all the stupid stuff out there and trying to figure out where I forgot a semicolon or a quote or a right angle bracket. The the rush comes from telling a computer what to do and it does it. I love that. So, anyways,

Craig Taylor (02:36)
Mm-hmm.

Justin Shelley (02:38)
I'm I'm I I'm nice to mine. I

Joshua Holloway (02:37)
Well, and getting to yell at it when it doesn't.

Craig Taylor (02:40)
Yeah.

Justin Shelley (02:41)
I I fully believe that they're gonna become autonomous and run our lives and I'm gonna be they're gonna love me.

You guys can be dicks to him if you want, but I will not do that.

Craig Taylor (02:49)
Joshua he's one of the guys that waste

tokens with please execute this command for me

Mario Zaki (02:54)
Yeah.

Justin Shelley (02:55)
no that yeah, that

Joshua Holloway (02:56)
And then when it gets that, it's

like, thank you so much. You did this almost okay. But if you are all right with my next suggestion,

could you please do the following?

Justin Shelley (03:04)
Please I have not not so many words, but please.

Craig Taylor (03:05)
Mm-hmm.

Mario Zaki (03:05)
Do you know if you curse at your

if you curse at your AI enough it starts cursing back at you? Did you know that?

Justin Shelley (03:11)
yeah.

I do know that. I I keep my models

Joshua Holloway (03:14)
No, that'll

Justin Shelley (03:15)
separate. I have one that I will swear at. My coding environment, I do not. I'm I'm very, very professional. And

Craig Taylor (03:20)
Mm-hmm.

Justin Shelley (03:21)
it it says, may I deploy once it's done running? I'm like, yes, please, you may. Whatever. Okay. Anyways, guys, way off track. Let's rein this shit in. We're gonna talk today about, hey, AI. What a surprise.

but we're gonna we're gonna go off. Actually, I've been teasing it. So if you guys are faithful listeners and you and you're always paying attention to what we're doing, you already know this. We're gonna bring this in-house because I'm really pissed off at vendors lately and I've been taking everything away from them, doing it myself, coding it myself, hosting it myself. And AI is the next one on the chopping block. And I'm probably pissing off all of my AI agents right now that I've rent. But anyway, sorry guys. don't worry. I still love you and I'll keep you around. But Craig is here.

Joshua Holloway (04:02)
They're gonna half ass code from

here on out.

Justin Shelley (04:05)
They're gonna half ass code. They better be doing more than half ass code. Cause I because

Joshua Holloway (04:07)
Yeah. No, they're gonna have fast code now 'cause you're trying to get rid of

Justin Shelley (04:11)
I I'm trash talking them. Probably. all right. So listen, we're gonna go around the room, do some quick introductions like we always do, keep them brief. Everybody knows who we are. Craig, you can talk a little bit more because you've only been on here. This is only your second time. and then we're gonna get into open weight AI models. That's the that's the I was gonna say a teaser, but it's not really a teaser, the introduction, whatever, Mario.

I'm gonna shut up now. Tell everybody who you are, why your head's bald and what you do, and then Josh and then Craig.

Mario Zaki (04:40)
Hi, Mario Zacchi, CEO of Mastec IT, located in New Jersey. I've been bald since I've been about seventeen years old. And

Justin Shelley (04:48)
Ha ha.

Mario Zaki (04:50)
we specialize in helping small businesses stay safe and helping business owners sleep better at night, knowing that their business will be there the next day.

Joshua Holloway (05:02)
Well, my name is Joshua Holloway. I'm the CEO for Seventh Eye Technology out of Sacramento. I was brought in because I have more hair than both Justin and Mario put together. no, but I'm the CEO for an MSP out

Mario Zaki (05:13)
We're shaving that next week, by the way.

Justin Shelley (05:14)
But exactly,

if you get drunk enough in Nashville, that's all coming off.

Mario Zaki (05:20)
Yes.

Craig Taylor (05:20)
Yeah.

Joshua Holloway (05:21)
Good luck.

but we're an MSV based on compliance services where businesses need to operate with compliance and we help them navigate how best to apply the the compliance, do their everyday, be able to use their tools, because to be fully compliant is to turn everything off and not do anything because that's how we protect all our data.

Craig Taylor (05:42)
There you go. Craig Taylor,

Justin Shelley (05:42)
Yeah, isn't that the truth? Craig.

Craig Taylor (05:45)
CEO of Cyber Hoot. My background is 30 years ago, I did a psychology degree and I tried to apply cybersecurity the traditional way to Cyber Hoot. Didn't work, so we pivoted to leverage psychology education in terms of changing people's behaviors.

And the only way to do that is not bigger sticks for clicks and to punish people for mistakes, which is what the industry seems hell bent on doing a little tunnel vision there. what does work though is positive reinforcement of good behaviors. You can train a dog with a shock collar and it'll quit on you. It'll stop participating and it will get afraid or it will bite you. Or you can train a dog with treats and it loves it and it participates and it learns new behaviors that it internalizes.

Let's stop treating our employees like their shock collars are on and more like we're gonna give them motivation and treats to change their cyber behaviors.

Mario Zaki (06:46)
I've tried both by the way.

Craig Taylor (06:48)
Mm-mm.

Mario Zaki (06:49)
one is better, one is not really necessarily better than the other.

Justin Shelley (06:53)
Well

Joshua Holloway (06:54)
Well and I'm wondering if you just came up with a new device.

Craig Taylor (06:55)
According to

your opinion or according to science? I know.

Mario Zaki (06:59)
No, I'm I'm just I'm just busting you.

Justin Shelley (07:02)
I I want a whole new

episode now on shock callers because I've got stuff to say about that, but we'll we'll save it for a different day. Guys

Craig Taylor (07:08)
That's a different

Joshua Holloway (07:08)
Is i is

that gonna be it N C seventeen?

Craig Taylor (07:09)
show, isn't it?

Yeah, it's the Dr. Grey show.

Justin Shelley (07:12)
yeah. yeah. We're not gonna talk anymore. That that's

it's actually a whole different podcast. I'll I'll drop the link later. We have in studio right

Craig Taylor (07:21)
Brian?

Justin Shelley (07:22)
now Brian LaChapelle. Brian, you're late. Say hi.

Bryan Lachapelle (07:26)
So I know. I

I got sidetracked. hi. I was at a conference.

Justin Shelley (07:31)
Well tell everybody who you are. Not

why you're late. Tell people who you are. Dude, we're doing introductions. Rapido. Okay.

Bryan Lachapelle (07:39)
everybody knows who I am. Brian Lashpaw,

B4 Networks, Niagara, Ontario, beautiful. I don't know.

Justin Shelley (07:46)
There go.

Good enough. 1% better every day. Welcome, Brian. let's we're gonna just jump right in.

Mario Zaki (07:51)
We're shaving his head next week too.

Justin Shelley (07:53)
So, Brian doesn't even know what

Craig Taylor (07:54)
Yeah.

Justin Shelley (07:55)
we're talking about today. We're we're talking about open weight AI models. Are you familiar with this, Brian? If not, start doing some research real fast because we're gonna jump in and I here's how we're gonna break this down. I want in in relatively simple terms, we don't want to get too technical. What is open weight?

How does it compare to open source? Are they the same thing or are they different? what are the advantages and disadvantages from a utility standpoint, from a cost standpoint, and from a security standpoint? There we go. There's the agenda. Craig's feverishly typing away. Craig, I thought you came prepared.

Craig Taylor (08:35)
I'm just asking novel questions because I

popping into my head about open weight models,

Justin Shelley (08:41)
Okay.

Craig Taylor (08:42)
another big thing that was asked today that I agree with, and I'll bring it up when it's right time.

Justin Shelley (08:47)
Okay. Well I'm I'm I'm making this open mic. Who wants to say what an open weight model is and is it any different from an open source?

'Cause nobody did their homework.

Joshua Holloway (09:01)
No, we did our own.

Justin Shelley (09:04)
Well somebody answered the question then.

Joshua Holloway (09:06)
All right, open weight means you download and run the weights yourself. the training data and the pipeline usually are not included, which means it is up to you as the user to train it, give it the information that it will learn from. And then open source LMs.

Justin Shelley (09:19)
So do we have to then like

rip all the books apart and shove into a a scanner and stuff? Is that is that what we're talking about?

Joshua Holloway (09:26)
Yeah, you're you're training it. And and unless you download one that has the training information in it. But if you want it just to learn solely off of your business, it's like ripping the covers off of the book. You're just giving it all your business information.

Craig Taylor (09:40)
but it's still on your private open weight model. It's not shared with OpenAI and Anthropic, right? The biggest mathematicians in the world were really pissed a few weeks ago when OpenAI solved the math, like think of the most classic difficult math problem that hasn't been solved. I don't know what it was called. I could look it up, but it was solved by OpenAI and hundreds of agents working thousands of hours. But when they looked at the proof,

They found some of the prompt and some of the answers that mathematicians of the world who are also trying to solve this math problem were using in their answers and in their prompts. And OpenAI was asked about that. Now I could substitute anthropic in the same way. And they said, we humans don't consume your prompts, but the AI can see it and learn from it. So they were taking collectively the world's knowledge from the smartest people.

and applying it to this math problem to solve. And they were successful. You how far we've come in three years when it couldn't do basic algebra.

Bryan Lachapelle (10:45)
It's

Justin Shelley (10:46)
Yeah, like basic story

Joshua Holloway (10:46)
You guys remember when

open AI couldn't even do any math? Yeah.

Justin Shelley (10:48)
problems. Yeah.

Craig Taylor (10:50)
Mm-hmm.

Bryan Lachapelle (10:51)
Couldn't e couldn't even tell how many R's in strawberry.

Justin Shelley (10:54)
I was gonna

say it still can't spell strawberry,

Craig Taylor (10:55)
Yeah.

Justin Shelley (10:56)
right, Mario?

Bryan Lachapelle (10:57)
Yeah.

Mario Zaki (10:58)
Yeah.

Craig Taylor (10:58)
I received

Mario Zaki (10:58)
I actually I haven't tested

Craig Taylor (10:58)
it 17.

Mario Zaki (10:59)
it recently. I saw there was another one, but I haven't tested that.

Craig Taylor (11:03)

Mario Zaki (11:04)
Yeah.

Justin Shelley (11:05)
Can't spell seventeen.

Mario Zaki (11:07)
it I I mean it's still ha how

Craig Taylor (11:08)
No, it thinks there's three.

Mario Zaki (11:10)
many times have you done something and you you're looking at it like this isn't correct? And it's like, you're absolutely right. I should have caught that. I'm like, Well, what the fuck?

Craig Taylor (11:21)
That's it.

Justin Shelley (11:21)
Alright, so back

Joshua Holloway (11:22)
Well that's just so

I can get your token

Justin Shelley (11:22)
on track.

Joshua Holloway (11:23)
spin.

Justin Shelley (11:24)
Back on track, guys. Open weight versus open source. Have we settled this yet? What's the difference?

I think we know what open weight is.

Craig Taylor (11:31)
So

can I throw in a throwback analogy? So when you're listening to this, you can think back to the 2000, when every company in the world was trying to get online and they're trying to figure out where do we host our data, our database, our web server and our information? Do we do a private web hosting hardware and software that we built ourselves? Because that's expensive, but it's safe because it's ours and we own it.

Or do we do a public one where it's like one server and we have 75 companies on it. One gets breached. My data is at risk. That's a public model. Then there's a hybrid model, which happened to win out over the day, which was AWS, Amazon, where you could have this shared infrastructure that is dedicated to you in a data center they built, but it hosts all your data. We have the exact same argument going on right now between OpenAI,

Anthropic, Muse, all these public LLMs that purport to save and protect your data, but do they? They keep your prompts for 30 days or private. Yeah, they do that.

Justin Shelley (12:37)
They call the cops on you if you say bad things. Somebody was threatening the sheriff or something the other day, I think in Florida, maybe. I don't know.

Craig Taylor (12:44)
Certainly. Or do you

Joshua Holloway (12:45)
It's always Florida.

Craig Taylor (12:46)
use the open weight models where you hosted on your own infrastructure, right? Somebody is going to come out in the middle and say, let me give you a mostly private. And I think some have like AWS Bedrock.

where you can, I consulted with a firm that did tabletops and they practiced with AI. The AI would listen to the tabletop and they said, we're putting our sensitive biggest mistakes and our critical gaps in our incident response into a public LLM. No, you can't. They said, you cannot do that. You have to put it on a private LLM. So they went back, retooled, put it on bedrock and it was much more amenable to the.

corporate clients wanting to run tabletops, because it just listened in to all the conversation and summarized who talked the most, who said they had an action item, who did this and all that, just like your digital assistants of your Zoom meetings and this meeting and that sort of thing. So that's the big conversations happening today from a cybersecurity perspective in my background, in my world, where how much data do we give these public LLMs?

Even if we buy the right license model that says they don't store our data, they don't consume it, they don't do it, there's already evidence that they're doing it.

Bryan Lachapelle (13:58)
Yeah. To to answer your question though, Justin, directly, open weight and open source both on your own hardware, one's trained, one's not.

Justin Shelley (14:07)
Okay. Thank you. ten minutes later. so what what is the training process?

Bryan Lachapelle (14:12)
I had to research it. No, I'm kidding.

Justin Shelley (14:18)
And listen, I'll straight up admit this. I have not hardly started digging into this at all. other than a little bit of prep for today's episode.

Mario Zaki (14:24)
So we we

so we actually have a server in-house that we're hosting and it was an open weight model where it was it's running on Hermes. We had to teach it everything. We had to pretty much give it access to our PSA and some tools and stuff like that, and we had it go and learn and it's constantly learning.

Okay. The it's learning at our pace, like how many tickets we you know, we put in per day. It's not taking any other people's information and help you know, teach it. It's only learning on everything that we've taught it and what we've told it you can start digesting and learning. So it's like a a a an empty model that it started with and we're constantly teaching it and when it's gets it's you know, gets something wrong.

We're going in there and like, no, this is the way you should be doing it because a lot of times we'll ask it a question like and then it'll you'll pretty much say, Shit if I know. You know, we're like we're like, all right, well, this is how you start learning, you know, and then we provide it more information.

Craig Taylor (15:38)
There are less safeguards and guard rails on the open weight models from what I understand. Have you seen any evidence of that, Mario, where it doesn't refuse to do something that, say, Claude or Anthropic would refuse to do, such as investigate these 17,000 events at Hugging Face?

Mario Zaki (15:56)
to be honest with you, we I I think we have the the the safeguards in place. We haven't actually tried to have it attack anybody. You know, it's still on its leash.

Justin Shelley (16:08)
Don't you dare t don't send it my way.

Joshua Holloway (16:08)
So typically the open weight models when you Yeah,

Craig Taylor (16:08)
Mm-hmm.

Joshua Holloway (16:11)
usually when you when you load an open weight, it has the guardrails turned on. You you have to go through it and turn them off. At least that that that's what I've read or I haven't

Craig Taylor (16:19)
Okay.

Joshua Holloway (16:22)
tried.

Justin Shelley (16:24)
Okay, so hey show of hands, who actually has one running? Mario, you've got something running in house. which which model are you using? And then does anybody else

Mario Zaki (16:31)
Hermes.

Justin Shelley (16:32)
have one? Hermes is the the

Joshua Holloway (16:34)
I have a l

a llama running on a a Mac and it's it's I think it's on GLM three is the one it's currently running. One of the versions of GLM because it's not a super powerful Mac. I mean we asked it a question, I think it it was trying to figure out what the capital of Illinois was for a good solid forty five minutes.

Craig Taylor (16:57)
Yeah.

Justin Shelley (16:58)
I was gonna ask that. Like what how how

Joshua Holloway (17:00)
Mm.

Justin Shelley (17:01)
limited is their knowledge? So it's it's

Bryan Lachapelle (17:02)
boy.

Justin Shelley (17:02)
really bad. I'm not I'm

Joshua Holloway (17:05)
That what that one

was Yeah.

Mario Zaki (17:07)
It it will actually take me longer than that because if I don't get to search it on the internet because I have no clue.

Justin Shelley (17:15)
I'm not super excited about open weight yet. I might be leaning open source 'cause it's already trained and it knows all this stuff, right?

Joshua Holloway (17:24)
Well it depends the hardware you throw at it, right? Because you you need the the GPUs to help with its crunching. And I like my my system is super low end. I and I did that intentionally. I wanted to

Craig Taylor (17:36)
you

Joshua Holloway (17:37)
see what would happen. Right. The more hardware you throw at it, it'll crunch through, it'll analyze faster. I primarily

Craig Taylor (17:40)
you

Joshua Holloway (17:43)
built it to test it for using heartbeats. So keeping agents alive early on where like open claw, you'd have heart beating and it to to keep the agent awake.

I

was using GLM three on O Lama, which was all free, just to do heartbeat checks. And it was efficient there because it's literally like millisec millisecond heartbeat checks to keep the agent awake. Other than that, I didn't use it for much of anything else. I couldn't get it to do much.

Mario Zaki (18:10)
Yeah, I had to get customer.

Justin Shelley (18:11)
No nobody else is running one.

Joshua Holloway (18:11)
Mm.

Bryan Lachapelle (18:15)
Not not not at this time.

Justin Shelley (18:16)
You know, I'm I you can run there's some of these you can run on a Raspberry Pi. That might be my first I have a little my little AI in a box that I carry around with me.

Bryan Lachapelle (18:26)
That'll just take a year to give you the the capital.

Joshua Holloway (18:28)
No.

Justin Shelley (18:31)
Okay, so Craig, you suggested this topic. So I want just bullet points on we're gonna talk about the the pros and cons from utility standpoint

security standpoint, cost standpoint of open weight, open source versus the stuff we pay for on rent.

Craig Taylor (18:54)
Mm hmm. So we know that open weight models are getting, there's pushback from Claude and Anthropic because it eats into their revenue streams, right? There's the model costs are a lot lower. There's the privacy issue of data governance. Where does the data live and who has access to it? That's another piece of

you know, anthropic and open AI are trying to assure us that they're not consuming our data. They're not learning from it, but we just had evidence

Justin Shelley (19:24)
Liars.

Craig Taylor (19:24)
in that math example.

Justin Shelley (19:26)
Mm-hmm.

Craig Taylor (19:28)
so the, the cost of operation is much lower potentially. And then there's the guard rail and security, right? If you look at the specifics of the hugging face incident, they first turned to anthropic or Claude and said, what's happening on our network? What are these 17,000 events? And it refused to help because

Justin Shelley (19:45)
Yeah.

Craig Taylor (19:46)
they were security.

attacks and malicious code was being executed in different ways and it was not participating. So they opened, I don't know which model it might've been, GLM in the five series. I don't know when that came out, but they had to turn to a Chinese model essentially to investigate the incident to determine, hey, we've been hacked by OpenAI. It's running these things on our network and then reached out.

So you have certain guardrails that you can erode or shut off in open weight models that can help you do more from a pen testing or an attack or offensive or defensive perspective.

Justin Shelley (20:33)
I mean it it's

an interesting I I cannot get behind the guardrails in the cybersecurity world because we have to fight with the same equipment and intelligence that our enemies are fighting with. Right. I I I can't go anywhere but there. And then at the same time, I can't get behind the idea of unleashing these models with no guardrails to the public. So I'm I I gotta admit, I'm really torn

Bryan Lachapelle (20:56)
Mm.

Craig Taylor (20:56)
you

Justin Shelley (20:57)
on this one. I don't I don't know. I don't know what to do. I w I trust

Craig Taylor (21:00)
Well...

Justin Shelley (21:01)
myself with it. I want it.

I want unguard railed everything for me, but nobody else. Thoughts.

Bryan Lachapelle (21:07)
Good luck with that.

Joshua Holloway (21:08)
Yeah. What

what do you do when it breaks out 'cause it's unguarded rail?

Justin Shelley (21:09)
What is

isn't that exactly what the the bigwigs are saying though? We're gonna control everybody else except but you can trust us, don't worry, we're we're the good guys.

Joshua Holloway (21:20)
But they proved that we couldn't because that's one of multiple incidences where it has broken out from open AI.

Craig Taylor (21:29)
from OpenAI, from Muse, from all of

Justin Shelley (21:31)
All of

Craig Taylor (21:31)
the big vendors. And I read something recently

Joshua Holloway (21:32)
Yeah, Claude, you name him.

Craig Taylor (21:36)
that there was one commonality to a lot of these events where the LLM broke out of its environment. And that commonality was the Israeli firm who they were paying to sandbox the LLM and test it.

and tested in ways that would exercise it without allowing it to break out and do these things. the testing environment was substandard, but there was one company that was hired by multiple AI vendors to do some testing on their

Justin Shelley (22:09)
Or

Craig Taylor (22:10)
model capabilities.

Justin Shelley (22:12)
Or the

commonality is everybody wanted the headline. Yeah, our AI is so powerful it broke out too. It's so strong.

Bryan Lachapelle (22:21)
I got one in my back pocket

Craig Taylor (22:22)
You're not wrong.

Bryan Lachapelle (22:23)
and it broke out too.

Joshua Holloway (22:25)
Mm-hmm.

Justin Shelley (22:26)
I don't know. I I I I hadn't heard that, Craig. That is an interesting point. I do also think there is definitely some hidden secret agendas. I've I've had my tenfoil hat on for a while. all right, so we've we've talked

Mario Zaki (22:41)
So Justin, I

mean, coming back to what you were talking about earlier and Vin talking about with you know, hating these SaaS companies, what's gonna end up happening is like I what I'm seeing is a security company that's gonna come out and says, you know, we're fighting fire with fire, we have ours you know, the guardrails are down, you know, and that's something that is gonna be very hard for you to kind of replicate on your own.

or any of us to replicate on our own if we don't have that readily available to us.

Justin Shelley (23:16)
I know we gotta get these models now while they're still available and unguard railed. Right? I mean, isn't that what you're saying? That's what I just heard. I'm

Mario Zaki (23:23)
Yeah.

Justin Shelley (23:27)
I haven't done it yet, guys, but I the I I don't know. I I have more questions than answers right now, but I am leaning heavily towards equipping myself before we get regulated out of it.

Craig Taylor (23:41)
See, there's a school of thought that it's already too late, that these models are already good enough and strong enough to be self-improving with the right training that you can do on them, that you're not gonna be able to contain that genie anymore. So

Justin Shelley (23:59)
Interesting.

Craig Taylor (24:01)
the horse has left the stable in a really old analogy that shouldn't really work anymore.

Joshua Holloway (24:06)
And the barn door's wide open.

Craig Taylor (24:08)
Yeah, exactly. In other words, you can't put it back. This Pandora's box has been opened and it ain't being closed.

Justin Shelley (24:16)
No, I agree.

Bryan Lachapelle (24:17)
I would agree

with that. Yeah. I would agree.

Craig Taylor (24:20)
That company in Israel was called Irregular, an Israeli AI security testing firm formerly known as Pattern Labs. And its evaluation infrastructure was linked to incidents involving OpenAI, Anthropic Meta, and later Google. And then the question that I think is on my mind is, why are we saying that the AI did it? The AI is doing what it's told.

Justin Shelley (24:43)
I know. Hey, I didn't do shit. Yeah.

Craig Taylor (24:46)
The companies that are testing these models set a goal

and the goal was achieved, it's just that, you you got to be careful what you wish for.

Justin Shelley (24:55)
It's it's all fear. It's

it's there's there's so many hidden agendas. We had an episode, Craig, where we just said follow the money. Yeah, and and there's that money's flowing in different rivers. but media media needs everybody to be afraid so that they'll listen to right? And so number one, these headlines have to say, my god, AI is gonna take over the world. It's gotta scare the shit out of us, or we're not gonna listen to And I think that there's a reasons why the the CEOs of these companies are doing the same thing. They they want that message out there.

They want us to be afraid. anyways, I don't want to repeat a whole previous episode, but let's let's talk a little bit about the cost. What is the the cost to like I said, you can run some of these on on Raspberry Pis. And I I haven't dove into this myself, but I just keep seeing more and more you know, the they're becoming more and more I'm I'm losing the word because this is what I do lately.

'Cause

I'm old. it's tangent while I'm trying to figure out the right word. I called my bank today. They wouldn't even talk to me. They kept saying, Press one if you're over fifty and it wouldn't let me do anything else. And so I did, and that's kept trying to give me away a free life alert thing.

Mario Zaki (26:06)
Yeah.

Justin Shelley (26:07)
And I couldn't get to talk to me about banking. It's all they want to talk about

Joshua Holloway (26:10)
Yeah.

Justin Shelley (26:10)
is life alert and insurance, and if I fall down I can't get up nonsense. That's that's kinda why I'm so h

Joshua Holloway (26:15)
That was a good tangent.

Justin Shelley (26:16)
I know, and I have no idea where I was going before, so whatever.

Joshua Holloway (26:20)
Mm mm.

Craig Taylor (26:20)
pricing

of open weight versus paid involved. And I think I saw Anthropi announced some price reductions. They're all trying to reduce prices to compete with these open models so that people don't go to them to then build infrastructure for them, to then adopt them and eat into the billions and billions of dollars they have to generate in revenue when they go public

Justin Shelley (26:40)
Just to break even.

Craig Taylor (26:42)
just to break even this year, yeah,

for their investments.

Justin Shelley (26:43)
Yeah. They're becoming more efficient.

That's what I was gonna say. Is as as they're becoming, you know, we're using more and more parameters. We're we're into the trillions now, right? but they're they're doing more with less resources. And so the while the LLMs are getting smarter and you know they're also doing it with less compute power. So right

Craig Taylor (27:04)
Mm-hmm.

Justin Shelley (27:05)
now today we can run one on a Raspberry Pi. It's sure not I'm sure it's not gonna be great, but we can do it. Actually they're they're putting them on these little tiny microchips that

Or nothing.

We still need bigger computers to do something more powerful, but I don't know how long it is. This is in my mind, this is a repeat of, you know, if we rewind technology twenty, twenty five years or thirty years, we're always excited about the new processor coming out or the the new video card or, you know, my God, my my motherboard can support sixty four K of RAM. You know, that kind of stuff. I think that's where we're going. We're we're just doing another version of that. What do you guys think?

Mario Zaki (27:40)
Yeah.

Yeah, I I

Joshua Holloway (27:41)
Yeah, you're definitely

Bryan Lachapelle (27:42)
Well it's the

Joshua Holloway (27:43)
yeah, definitely.

Bryan Lachapelle (27:43)
Yeah.

It's the whole mainframe and and and PC

Justin Shelley (27:46)
Yeah.

Bryan Lachapelle (27:47)
argument and all over again, right? So, you know, right now the big models are all being hosted on massive data centers, but as the technology gets much better and you know, I can imagine down the road, you know, we'll be able to work on on or that or or our desktops will get so good that, you know, we'll be able to run it locally and not have to worry about, you know

Basically them being on on the mainframes or the the the cloud hosted. It'll be it it goes in circles. It just keeps coming back around and around.

Joshua Holloway (28:19)
Well, and I I I agree

Bryan Lachapelle (28:20)
You should call it something different.

Joshua Holloway (28:21)
with Brian absolutely because right now the other thing that we're seeing too is cost of hardware, right? Availability and cost. memory,

Justin Shelley (28:28)
Through the roof.

Craig Taylor (28:28)
Mm-hmm.

Joshua Holloway (28:29)
yeah, it's through the roof. Memory sticks are at astronomically high. And getting equipment is months on end. We have a a server we had to purchase for somebody, six month waiting list.

Craig Taylor (28:41)
Mm-hmm.

Joshua Holloway (28:41)
And you had to approve the PO and pay a certain percentage and then wait for

Justin Shelley (28:46)
Just to get on the list.

Joshua Holloway (28:47)
the hardware. Yep.

Justin Shelley (28:49)
I'm I'm quoting

Joshua Holloway (28:49)
Yeah, and and

Justin Shelley (28:49)
one right now, and I keep telling you guys I've got a source you need to check into. I can get the hardware, but I mean the price, the the RAM just for a normal amount of RAM for a server is more than half of the price of the entire unit.

Bryan Lachapelle (29:02)
Mm.

Justin Shelley (29:03)
It's crazy. Like I think I'm at like six thousand dollars for RAM just on and it's a small build. This isn't even a big powerful computer. whatever.

All right.

Joshua Holloway (29:12)
So those are start

there's the start of your cost right there for bringing things like that

Justin Shelley (29:14)
Yeah, yeah.

Joshua Holloway (29:16)
in-house, right? So it's the same thing with when cloud started to become popular and there were the naysayers where

keep everything in in-house. Then you had some guys that were kind of hybrid, they'll dance between the two. And then you had the full end, jump in, all cloud, right? And it and it diversified your costs. I think for a while we even saw the price of hardware come down for a while just because people were dancing between the AI is the same thing. It's there's going to be business cases where you have to have a high in-house. It will either be because you have a compliance that you have to follow, you can't share data or

You have an executive team that is super afraid of the fact that their data may be training a model, shared inside a model, or may leak. So they're going to want to bring it in-house and ultimately they're going to have a higher upfront cost because they have to get that hot hardware and bring it in and then and then run it, versus the guys who are like, we don't really care too much. We'll go to an anthropic, we'll go to an open AI or any other models that are out there and we'll use API keys and and communicate. Or you'll have that kind of hybrid where somebody will.

utilize AWS or Azure and and have their models inside of there. So they kind of get the best of both words in a sense. So I mean that's ultimately what you're talking about. And the price

Mario Zaki (30:32)
Yeah. Yeah.

Joshua Holloway (30:35)
the price is probably the same when you do the calculations over time. It's just upfront costs and you're gonna have spikes every so many years to replace the hardware, then that's bringing it in house or a flat budget across the board because you're hosting it through a cloud provider or

An open AI model system.

Mario Zaki (30:54)
Yeah, and we're kind of in

that hybrid model. Like I mentioned earlier, we have an in-house server that we we used on like a a server that we were having that we just had sitting around. We did get a Tesla, GPU that put in that I forgot how much that was. It's like seven hundred dollars or something like that, that we put in. So it was an upfront cost, but we have it kind of doing certain things, not everything for us. And we still have subscriptions to God, I have like

Pod, I have cursor, I have you know, some other stuff in there. So we're kind of doing the the hybrid. But one of my guys wanted to just say, well, why don't we just bring like put in a whole big one in there and put like seven GPUs in I priced it out. It was gonna be like thirty-five thousand dollars. I'm like, I'm

Justin Shelley (31:46)
Yeah.

Mario Zaki (31:46)
fine with the monthly subscriptions, you know, like

Craig Taylor (31:49)
Mm-hmm.

Mario Zaki (31:49)
you know, the

Because the by the time you even come close to breaking even, all the models and all the stuff that you have in there is gonna be obsolete. Very similar to what you guys were talking about. You know, a long time ago, you get a new processor or whatever, you take it home, it's already obsolete.

Justin Shelley (32:07)
Right. Yeah.

Mario Zaki (32:09)
Yeah.

Justin Shelley (32:11)
All right. Well have we answered any questions today?

Mario Zaki (32:15)
Not

one.

Justin Shelley (32:16)
Think we have. This is gonna be the episode

Mario Zaki (32:17)
Yeah.

Justin Shelley (32:18)
that like resolved nothing. Craig, I'm putting it back on you. You you propose this topic. We're we're gonna kind of

Craig Taylor (32:25)
Sure.

Justin Shelley (32:26)
wind it down because I think we've just we've said what we can say, and and the reality is as I see it, is there is this great big unknown. you know, it and it's it's such a weird time that we're in in the world of technology because it just changes not even every every week, it's like every day or every minute there's there's new information.

So what's what's your take? let's let's kind of summarize the whole thing. Your take on these open weight models. What what what interested you and what made you wanna even talk about it today?

Craig Taylor (32:58)
So for me, it's all about the data, right? When you're talking about your business, you have to know what kind of data you have and what you're putting in and what you're using it for. If you have critical or regulated or defense contract, FCI, CUI, you have to be 100 % certain where that data is living and what you're doing with it. And if you have that level of criticality in your data, an open weight model privately hosted is probably the only thing you...

legitimately can do and manipulate that data well. Healthcare data, student data, financial data, the list goes on and on. You've got to be really careful. And that's where these open weight models and private hosting of them and training of them to suit your specific business needs and solve your problems can be really advantageous over the long run. But you've got financial pressures on you, right? You can maybe do it cheaper in these public models if you can

by the right license that doesn't let the data get consumed. I don't trust that. So I've never advised using them for those sensitive data. And then you've got your employees who need training on who can put what data into what models for what purposes. It's better in those scenarios where you have all that sensitive data in your business to just put it in-house than any mistake is in and contained, right? So I think there's...

There's this whole full circle. We've done this digital transformation, pushing everything out into the cloud, even the sensitive data for accounting firms, law firms, et cetera. And I see a future where there's going to be competition between these private weight models that you can host yourself. You can keep them contained and you can prevent. There's DNS methods, there's other tools and vendors popping up to say, okay, block access to all these other models.

because people are going to do the easiest thing. They're going to pull out their phone, they're going to do what they need to do to get the job done. And you just want to provide them tools that work and speed up their jobs without putting your business at risk of a breach or data exfiltration event or exposure.

Mario Zaki (35:08)
Now let me ask you a question. or also this can go to Josh too, since you guys do compliance a lot. If somebody were to enter like

Craig Taylor (35:16)
and

Mario Zaki (35:17)
a credit card statement into Claude and say, Hey, tell me, you know, what my biggest expenses were for, you know, September, it's not like somebody else is somebody's gonna say, Hey, you know, s send me Mario, you know, what is Mario's credit card number? You know, like

What what exactly is gonna happen if you end up putting some of that stuff up?

Joshua Holloway (35:43)
Well,

depending on how the data's stored and how you prompted it in there, like case in point, a lot of Claude's prompts are Google searchable because you've shared that prompt or that conversation. So if you accidentally shared that information, Google's had the ability to crawl it and it's available. you can do a general search on Google, like show me all the Claude prompts where people are talking about their credit card statements. You know, you could do generality searches like that. You're definitely getting

Bryan Lachapelle (36:11)
Give me a second.

Joshua Holloway (36:12)
You're definitely getting

Justin Shelley (36:13)
Ha

Joshua Holloway (36:13)
Yeah, yeah.

Justin Shelley (36:13)
ha ha

Joshua Holloway (36:14)
What's Josh's

Craig Taylor (36:14)
Yeah.

Joshua Holloway (36:15)
cr what's Josh's credit card number? yeah. And I think the biggest thing is

Craig Taylor (36:17)
We used to call that Google dorking, right? There's a whole book by Johnny Long about that.

Bryan Lachapelle (36:19)
Yeah.

Joshua Holloway (36:23)
is if it's a a commercial source, you should consider it like compromise. I wanna I wanna act in the most securest method possible, right? you know, it's what Craig was saying as far as being, you know, bringing in open weighted models and everything. There's one other one other caveat I wanted to hit on is there's not just

Or Western-based flavor of these open-weighted models. There's also Chinese-based. And if you're dealing

Craig Taylor (36:48)
Mm-hmm.

Joshua Holloway (36:49)
with CUI, God forbid they find out that you're using DeepSpeak to analyze your data because that's a Chinese model, right? So you have to be very careful on the models that you choose based on the compliances that you're governed by, because you can immediately just wash away all that compliance and consider yourself wide open or in trouble.

Justin Shelley (37:06)
I got a question

on that, Josh, before we get off of it. The and and I say CUI, you say C UI, but I've never heard anybody spell it out before. Can you use Chinese models if they're hosted locally? That still be compliant?

Really? And you're positive?

Joshua Holloway (37:27)
I'm fairly positive on this one.

Justin Shelley (37:29)
Interesting. Because your data doesn't leave. Your the model lives there and it can't talk to anything else.

They

Joshua Holloway (37:35)
Supposedly.

Craig Taylor (37:35)
Well,

you can take apart just about any Chinese manufactured piece of hardware, whether it's a net

Justin Shelley (37:41)
Right. And it's it's phoning home.

Craig Taylor (37:44)
gear router and it's got Wi-Fi chips and buses are built with Wi-Fi chips nobody knows about. And that's why the Trump administration banned all those things from import into the United States because they were taking them apart saying, OK, why is it phoning home? What is it doing here? TVs, you name it. there's there's in talking an open weight model, it's

trained on data, you ask it about Tiananmen Square in 1989 and it won't answer you in these Chinese models. They don't have the pantheon of all knowledge. They've been filtered and cleansed of certain things. What else have they done? I don't think we quite rightly understand it just yet.

Bryan Lachapelle (38:22)
You ask it

to code something, it programs in a back end call. Back

Craig Taylor (38:26)
Yeah, there are back

Joshua Holloway (38:26)
So

Craig Taylor (38:27)
doors into a lot of these chipsets and other things. Go ahead.

Justin Shelley (38:29)
On on the

hardware, yeah, so I'm I'm just curious on the software. We're

Craig Taylor (38:33)
Ahem.

Justin Shelley (38:33)
just talking about the model itself. If your hardware is good, doesn't have any Chinese spies built into it, the model itself

I is it can it? I don't think it can actually just send information out, the model itself, right?

Joshua Holloway (38:50)
Why wouldn't it be able to?

Mario Zaki (38:52)
Yeah, what makes you think that?

Craig Taylor (38:55)
How many outbound firewalls have blocks on all ports and protocols outbound that you've witnessed? I've never seen one. It's best practice. Inbound, sure, yes. We're not gonna let anything into our network.

Justin Shelley (39:06)
Because the model itself is just math,

right? There are no

Joshua Holloway (39:10)
Okay.

Justin Shelley (39:10)
instructions. The model itself is just math. Then you have to host it on a a container or whatever it's called. I I I'm gonna mess up the terminology here, but it has to run inside of software. That could phone home. That software has to live on top of hardware, that could phone home. But the model itself, if it's a Chinese model and everything else is fine, does it still violate the CMMC regs?

Joshua Holloway (39:37)
There's a level of uncertainty. Like if you're downloading

Justin Shelley (39:39)
Yeah.

Joshua Holloway (39:39)
a Chinese base model, let's say it is completely like not trained. There's nothing in it. And you're starting from

Justin Shelley (39:49)
Just the math.

Joshua Holloway (39:50)
scratch, just just just the math, gray area. I would still say you fall on the sword and say it is absolutely positively not allowed. But okay, maybe it doesn't know how to phone home. But you're if you're taking a model that has been trained.

Who knows what the training is? We don't know how it was trained. Maybe it is trained to be solely, you know, like everything for, and I hate to say it this way, like the motherland, right? So I'm gonna gather as much data as I can and I'm gonna send it home. And and I'm I'm I'm required to do that. and I wanted to bring up another point too. I I built a computer, a Mac Mini using and I put Quinn on it. And as I was talking to the LLM in English.

It switched over and started talking to me in Mandarin. All man all all all Mandarin code,

Justin Shelley (40:39)
Wow.

Joshua Holloway (40:40)
all Mandarin responses. And I was like, Hold up, wait a minute. Like, I was talking to you in English. We were we were working out something in English. Why did you switch? Yeah, we w we

Justin Shelley (40:51)
We were vibing, bro. What happened? What the

Joshua Holloway (40:55)
were doing good and you switched, Dobbby. And it was like, wait, you're right.

Justin Shelley (40:58)
nay.

Joshua Holloway (40:59)
I we were speaking in English, but originally I was trained.

you know, on this language and I I I I diverted back to my original training. That's pretty much what it told me. And I was like, we weren't talking about anything in depth. Like like I don't know why you got lost in this one. You know, so

Justin Shelley (41:15)
That's crazy.

Joshua Holloway (41:17)
you just never know

Mario Zaki (41:17)
I mean

Joshua Holloway (41:18)
what it's gonna do.

Mario Zaki (41:19)
I mean it it it's cheap for a reason and you know and and Justin

I'm gonna quote you i if something

Justin Shelley (41:23)
yeah.

Mario Zaki (41:26)
is cheap or something if a product is free, guess what? You're the product. Exactly.

Justin Shelley (41:29)
You are the product. Yeah.

Bryan Lachapelle (41:29)
You are the product.

Joshua Holloway (41:30)
You're the

Craig Taylor (41:31)
That's right.

Joshua Holloway (41:32)
Yeah, you are the product.

Justin Shelley (41:34)
Listen, I I

Craig Taylor (41:34)
So when you...

Justin Shelley (41:35)
I admitted from the beginning, I'm coming into this episode with questions. I have not started diving into this, but guess what my new obsession is? Fuck.

Mario Zaki (41:43)
I mean here's the thing.

We we need somebody, either one of you guys, because I'm not doing it, one of you guys, go ahead and try it, or one of our listeners, go ahead and try it and get back to us and go to unhackmybusiness

Justin Shelley (41:54)
I'm gonna do it.

Mario Zaki (41:55)
dot com and let us know what you come out.

Justin Shelley (41:58)
I'm I'm gonna do watch by next week. unhackmybusiness.com is all in Mandarin. No, I'm gonna I'm I'm gonna

Craig Taylor (42:04)
He

Mario Zaki (42:04)
It's gonna get hacked.

Justin Shelley (42:06)
do it, but I'm gonna sandbox it and or air gap it or whatever you this thing's not gonna be on the internet. It's not gonna be connected to anything except for like the little box that I build and a keyboard. Like I'm not gonna let it touch anything.

Craig Taylor (42:17)
Until when

are what are the intills and butts until you want to upgrade to?

Justin Shelley (42:21)
Nothing, period. I'm not

I'm not I I will download one of these Chinese models. I will not let it touch anything. That's what I'm gonna do. Yeah. Yeah. Nope.

Craig Taylor (42:28)
And so you'll never upgrade it. Yeah, put it in a Faraday cage. Don't ever upgrade it. So it's always going to

Joshua Holloway (42:29)
You gonna put it in a Faraday cage?

Craig Taylor (42:34)
be that model forever and ever. And you'll never let DNS ping packets out. It can't communicate on those. And yeah.

Justin Shelley (42:41)
No internet. I

just want to be able to talk to it. I don't know. I'm gonna I'm gonna we're gonna play around. It's time to start experimenting. I don't know. Guy.

Mario Zaki (42:48)
I mean

add a Wi Fi card on it

Bryan Lachapelle (42:53)
Yeah.

Mario Zaki (42:54)
and say if you want

Justin Shelley (42:55)
No.

Mario Zaki (42:55)
to get on the Wi Fi, figure out my Wi Fi password and let's l let's see

Justin Shelley (42:59)
No.

Mario Zaki (43:00)
if it if it does it.

Justin Shelley (43:02)
And only use it at public Wi-Fi locations. Never from my network. So I can't be tracked. Yeah, okay.

Mario Zaki (43:05)
Exactly.

Justin Shelley (43:11)
I don't know. I've I've I've got ideas, but I don't have a plan. Let's say that. What else, guys? We're gonna we're gonna kind of move to wrap this thing up. like I said, I think I've I'm leaving with more questions and answers, but I do have one more thing to distract me from my real work.

FML. what else guys? Let's let's go ahead and just go around the room, key takeaways, final thoughts, say goodbye, whatever. You could take a a minute or two each and and then we're gonna we're gonna close it out. Brian, you've been quiet, so I'm gonna put this on you first, and you guys have to keep track of the order. Brian, then Craig, then Mario, then Josh. Can you guys remember that? Go ahead, Brian.

Bryan Lachapelle (43:53)
Yeah,

I don't have a whole lot to add to the conversation. I would I would just stress again that you know if you are going to be diving into hosting your own LLMs and your own infrastructure or even trying some of these open weight and on and or open source solutions, just know what you're doing and and experiment, but do it without using your actual

data first, right? Like just isolate it, keep it keep it separate and and then let me know how you make out 'cause I I'm not I'm not gonna bother.

Justin Shelley (44:30)
You're not doing it.

Bryan Lachapelle (44:32)
I'm not doing it. That's all I got.

Craig Taylor (44:36)
For me, think

Justin Shelley (44:36)
All right, Craig.

Craig Taylor (44:37)
the open weight models are making me keep me up at night because they're starting to approach some of the cyber, they're closing the cyber capability gap between Claude Mythos, Microsoft M-Dash and some of the open AI models around pen testing and zero-day vulnerability discovery. And when those open models have the equivalence, which we're seeing it's very close today with GLM 5.3,

we're going to see more companies and more infrastructure being breached throughout the world because not everything's been patched yet. Not everything's been scanned yet. We know if you look at Microsoft, they've been scanning the crap out of their software and they've been releasing thousands of patches, which

Justin Shelley (45:22)
Yeah.

Craig Taylor (45:23)
they wouldn't have done in three years of time. Two years ago, they couldn't have done thousands of patches, but they've been doing it because these human code that...

We've got this huge tech debt in the world of the software we've written and humans can't find the vulnerabilities that vary readily, but the AI models can. And when a GLM 5.3 points at some novel business that hasn't been scanned, hasn't been patched, hasn't had it zero days identified, all hell's going to break loose for a period of time

Justin Shelley (45:54)
Yeah.

Craig Taylor (45:54)
until we get rid of that tech debt in software. So that's what I think is coming. I was on a call today.

talking about Honeypot deployment for my virtual CISO clients because I want that canary to fall over

Justin Shelley (46:07)
Yeah.

Craig Taylor (46:08)
when someone gets in there, runs an Nmap scan and they say, okay, we've got a zero day attack going on. We got somebody in the network, let's get them out before the damage is done.

Justin Shelley (46:16)
Those are becoming really

important. I I think those should be more standard than they are. That it's very rare that anybody deploys those, but I'm with you on that. All right, Mario,

Craig Taylor (46:26)
Yep.

Justin Shelley (46:26)
final thoughts, closing arguments.

Mario Zaki (46:29)
Yeah, I mean, so w we we're using both models. You know, we're using our own server and it it runs well, but it has taken a a while for us to teach it and and then go in there and kind of provide it with more information and it learns and it goes at a s much slower pace and it's it's it it's dependent on what we're providing to it, you know, because it's

Out the door or you know, from the very beginning, it's like I can't do this. So we have to teach it and tell it. So you have to be very patient, you know, to do this, where the majority of people are not patient enough, especially when they're when they hear and see all the stuff that AI is doing. They want to go, you know, faster and harder and bigger and stuff like that. This isn't the way to do it if you're going

Craig Taylor (47:19)
Ahem.

Mario Zaki (47:20)
with a blank slate. but

It is it is definitely the safer way to do it, but slow. You know, so you have to be patient, you have to dedicate some time to teach it to ex to do exactly what you you you know what you want it to do. It's not gonna do the exact same thing as going right through a website or opening up something that is connecting through an API. So it's worth it if you want to dedicate, you know, put the time into it, or you have to.

Joshua Holloway (47:56)
And I and I think probably w what's gonna happen is people are gonna have to weigh the requirement to have a model in-house versus an external source. Obviously, compliance rings a bell here where it's like, do we have to bring it in-house? I think when it comes to financial, medical, obviously DOW, anything that is highly sensitive information, you're protecting the identities of people, state secrets, things like that.

You're gonna have to bring it in-house, but you're gonna they're gonna have to sit down, figure it out, and it is gonna be slow going. I've seen bringing a model in-house in in the aerospace sector and how slow going that was. And they just wanted to open up the door and they just wanted it to do things, and it got scary and it did some things it wasn't supposed to, but you you gotta go slow, you gotta prom you gotta plan properly, just like software dev. I think the other key takeaway to this one is the importance of cybersecurity in the sense that.

you know, NVIDIA released a tool for scanning skills and and and auditing just skills, skills that we can download and put in various LLMs. And it tells you exactly what it's doing. Is it doing something nefarious or could be nefarious? Like we have to, as like IT providers and cybersecurity experts, educate the our clients and and understanding that hey, you can't just download a repo or a skill and just run with it because you you think that.

It's cool. Everybody talked about how great it was. A person on Instagram said you should use it. you you got you gotta run it through some level of of vetting, right? And that's the reason why Nvidia released their their school, their skill scanner. The other thing that I'm starting to see pop up is penetration testing for vibe-coded apps, penetration testing for you know hosted systems and things like that. This is this is a the shift, right? We're going to

AI models that are breaking out and they're and they're finding ways to break into systems, whether it was planned, not planned, or anything along those lines. We have to come up to that same level and start to do penetration testing with models with the guardrails turned off or figure out a way to do that as secure as possible and find businesses that, you know, want that same thing because they want to better understand that their switches that they haven't patched in seven years are super vulnerable.

Craig Taylor (50:17)
Yeah

Joshua Holloway (50:18)
Or

the the their their firewall that they thought the firmware was being updated every month hasn't had a firmware update since like I don't know, 2018. but they keep paying for

Craig Taylor (50:28)
you

Joshua Holloway (50:28)
the licensing. You know, it's it's things like that. Like that's where we need to do better and we need to start doing those checks. And having the only way to do it is an open weight model or a service somebody builds that we and I I hate to say another SaaS application that we pay for.

Something like that we would have to get and then begin to use with our clients. And that's just the shift, right? Start to scan the skills and the things that humans are building, start to scan the networks and the systems. And it's it's just adding to our penetration testing and vulnerability scanning.

Craig Taylor (51:00)
Can I add one more suggestion in the world we're moving rapidly towards? was a Five Eyes advisory that gave some advice for these months we had before the open weight models and the frontier models would allow breaches to start happening in large amounts. Another thing I've witnessed this year is companies that have all their data ever produced on their internal networks still live and active.

And one of the best things you can do as a business owner or as an MSP is guide your clients to archiving as much of that data off the network as possible. We had a breach at a law firm and they had 25 years of client data that was exfiltrated. And we're like, why? Now you got to go find those 25 years of clients

Justin Shelley (51:47)
Wow.

Craig Taylor (51:48)
and notify them and apologize. And they've moved. They're no longer in business. The contact records have changed. If you just culled

and remove 20 years of data for the last five that you had. Put the other on a worm drive, put it in a safety deposit box, put it in a fireproof safe. If you need it, you've got it, but it's not at risk of a compromise that will lead to so much more pain and suffering. And companies aren't doing that. They don't have

Bryan Lachapelle (52:16)
Good point.

Craig Taylor (52:16)
good data management.

Justin Shelley (52:18)
Yeah. I it's something you see all

Joshua Holloway (52:20)
Well, definitely a good point.

Justin Shelley (52:22)
the time. You when

Craig Taylor (52:22)
Mm-mm.

Justin Shelley (52:23)
you g w anytime we go and

know, do a I'm saying investigation. That's the wrong word. an assessment or

Craig Taylor (52:30)
an assessment. Assessment.

Justin Shelley (52:33)
yeah, discovery, whatever on a on a potential new client and you open up their their server or wherever they're storing their files. It's just like, my God.

Craig Taylor (52:42)
Mm-hmm.

Justin Shelley (52:43)
yeah. That that needs to be cleaned up, but I'll tell you, Craig, I don't see it happening very often. I don't know who's doing it. you need to put it on the list of the hundred thousand things you're supposed to be doing as a business owner. I don't know.

Maybe, maybe it it's not even part of a framework anywhere that I'm aware of. So good God. There's so much we've got to worry about. Here's what I'm gonna close with. And I think I've mentioned this before, but you know, I I am I am very I want to say angry. I'm distrusting. I'm I'm I have a lot of anxiety around vendors right now. and especially as this thing keeps ramping up and changing every few minutes, we're we're dealing with something new. I was testing a new tool.

And it it's it's all still in beta, whatever. and it has AI built into it. And here's the problem that I've I I openly admit this in when I chat with AI, whatever platform it is, I believe those conversations to be somewhat private. Now I understand that they can be subpoenaed or whatever else, right? But in my mind, the the worst case was they might train on your data.

Right? And somehow it escapes that way. this product that I was testing, all of a sudden, something that I had put in an AI chat showed up in their social media marketing. Redacted, but without my permission, they're fucking reading my chats.

Craig Taylor (54:13)
Mm-hmm.

Justin Shelley (54:13)
And they they just took it and put it out onto social media. Not a word to me. I like, god damn it.

I am so sick of vendors right now. So

Craig Taylor (54:22)
Yeah. Yeah.

Justin Shelley (54:24)
yeah, am I gonna bring do I I don't trust Chinese anything, I'll say it, but do I trust my containment of a Chinese model more than I trust a goddamn vendor who tells me that they're secure? Right now I do. And so I d I don't know. I'm

Craig Taylor (54:36)
Well, did you,

Justin Shelley (54:38)
what

Craig Taylor (54:38)
did you read their privacy policy? I put,

I was so frustrated like you with, with vendors. went and started putting their privacy policies

Justin Shelley (54:47)
You've got a tool.

Craig Taylor (54:48)
through my AI policy scan website, app.aipolicyscan.com. And we started running all of these digital assistants. Think of Fireflies and, what was the one today? Read.ai and.

Otter.ai, we put their privacy policies through and their old privacy policies, we actually have an archive of these now and we're collecting them over time so we can see if they get better or worse over time. And we can explain six parameters that you should be looking at, whether data confidentiality, LLM consumption, marketing and sales of your data, et cetera, et cetera. And to the earlier comment, if it's free, you're the product. Well, they are putting in their privacy policies that we will improve our product by consuming

the conversations that are going into us. What does that mean? Well, one could argue selling to marketing firms the data to get money to improve the product is following that advice that they have in there. They are full, the gamut is full across the board of these privacy policies coming clean with what they're actually doing with the data. So I would go to that vendor that you were talking about, check their privacy policy on our website and see what it says.

Justin Shelley (55:59)
Give me your site again, Craig. You told me about

it and I forgot all about this. What what was the website?

Craig Taylor (56:04)
It's the site is called AI app, app.ai policy scan.com. there's pricing, there's how to work and get started as free for 30 days. we have a bunch of policies in there. ran one today for read.ai joined a meeting. was on, was like, that's not good. We've got to kick it out. And the guy was like, no, I want it here. And then I ran it as 3.8 out of 10 at three for confidentiality of data. had three for data sharing. had three for security commitments. And my gosh.

The purpose of this is to hold vendors accountable, to put them on the hot seat for what they're doing with our data.

Justin Shelley (56:42)
Yeah, but your website's not not working for me. Say it again or or link it in chat or something.

Craig Taylor (56:47)
I'll put it in the chat, how's that?

Justin Shelley (56:48)
Cause I'll I'll put this on the show notes. Guys, go to unhackmybusiness.com since I can't find this website. I'm not gonna expect you to, but go go to

Craig Taylor (56:55)
Where's the chat here? I gotta put the chat on here. It's not off of cyber.

This is a separate one altogether. Where is the chat here? There it is.

Justin Shelley (57:06)
Just go to unhackmybusiness.com episode one seven and I will I will

Craig Taylor (57:10)
Ahem.

Justin Shelley (57:10)
link this site once we get it figured out because I can't make it come up. app dot ai policyscan.com. I swear to god, that's what I yeah, it's not it's not working, my friend. Hmm, can't reach this site with a with a stormy cloud.

Craig Taylor (57:24)
Hmm. I wonder why that would be. Where are you located?

Joshua Holloway (57:30)
Hey,

Justin Shelley (57:30)
Nevada.

Joshua Holloway (57:31)
put on that that

Craig Taylor (57:31)
I'm on it right now.

Joshua Holloway (57:32)
that VPN. Put yourself over in like Ukraine or

Mario Zaki (57:34)
Yeah.

Joshua Holloway (57:34)
something and see if you can get to it.

Justin Shelley (57:35)
Yeah,

you well, at least I've got the right side. I'll still link it, but you you better go hurry and fix it because we're gonna drive a bunch of traffic to it. So

Craig Taylor (57:44)
Gotcha.

Justin Shelley (57:45)
all right, guys. With that, we are wrapping up this week's episode. I personally I don't like scaring the shit out of people, but I am currently scared. I don't know what's gonna happen. I I know that I'm I'm mad at Benders, I'm bringing some of this in-house.

And eventually it's probably gonna go back the other direction. Whatever. but I wanna hear your thoughts. Go to unhackmybusiness.com and give us your feedback there. But I know you're not going to, you're gonna get on social media. YouTube is where everybody's giving us comments right now. So fine, do that. Tell us what you think, what what the future holds, and what's your plan? And what is your tinfoil hat and crystal ball and all that telling you to do with it? Guys, we are going to wrap this up.

Well just as soon as I find the outro music, so I can just cut myself off by by playing it. So I have nothing. There we go.

Mario Zaki (58:44)
Unhacked.

Joshua Holloway (58:44)
Unhacked.

Bryan Lachapelle (58:46)
Unhacked.

Justin Shelley (58:46)
Mm.