Open Weight AI Models: Should You Host Your Own AI? - with Craig Taylor | UnHacked Ep. 107
Hosts:
Justin Shelley - https://www.phoenixitadvisors.com/
Mario Zaki - https://www.mazteck.com/
Bryan Lachapelle - https://www.b4networks.ca/
Joshua Holloway - https://7thdi.com/
Should you host your own AI so sensitive client data never leaves your building? Five security pros weigh cost, risk, and trust.
Justin Shelley - https://www.phoenixitadvisors.com/
Mario Zaki - https://www.mazteck.com/
Bryan Lachapelle - https://www.b4networks.ca/
Joshua Holloway - https://7thdi.com/
Should you host your own AI so sensitive client data never leaves your building? Five security pros weigh cost, risk, and trust.
Justin Shelley admits up front that he is coming in with more questions than answers. He is fed up with vendors and wants to bring AI in-house. So he asks a panel of IT and security people whether open weight AI models are the way out. Nobody agrees, and the disagreements are where the useful parts are.
The group starts with the plain-English difference between open weight and open source models. Then they get practical. Mario Zaki describes the in-house server his team trained on its own PSA data, and why he turned down a roughly $35,000 seven-GPU build. Joshua Holloway explains why a Chinese-trained model is a compliance problem for anyone handling CUI. He also tells the story of a model that switched from English to Mandarin mid-conversation. Craig Taylor lays out why regulated data (healthcare, financial, student, defense contract) is the strongest case for private hosting. He also explains why he doesn't trust "we don't train on your data" promises from public AI vendors.
It ends on two things you can act on this week. Craig describes a law firm breach that exposed 25 years of client data, and what it would have taken to avoid it. Justin tells how something he typed into a vendor's AI chat showed up, redacted, in that vendor's social media marketing.
ABOUT THE GUEST
Craig Taylor is CEO and co-founder of CyberHoot, a security awareness training company. He has a psychology degree and built CyberHoot around rewarding good cyber behavior instead of punishing mistakes. He also runs AI Policy Scan, a tool he built to score the privacy policies of AI vendors, and he works with virtual CISO clients.
WHAT YOU'LL LEARN
- The difference between open weight and open source AI models, in plain English, and why neither is a magic privacy fix.
- What it really costs to bring AI in-house: a Tesla GPU at roughly $700, a $35,000 seven-GPU quote, about $6,000 in RAM for a small server build, and a six-month wait on one server order.
- Why Chinese-trained models are a compliance risk for CUI and CMMC, and where the group disagreed about whether hosting one locally makes it safe.
- Why Craig says public vs. private vs. hybrid AI hosting is the same debate as web hosting in the 2000s, and how a tabletop exercise firm moved to a private setup.
- The data-hygiene move that could have limited a 25-year client data breach: archive old records off your network.
- How to check what an AI notetaker or assistant does with your conversations before it joins your next meeting. Craig scored one vendor's privacy policy 3.8 out of 10.
SUBSCRIBE
New episodes every week that explain AI, cybersecurity, and digital risk in plain language for business owners. Subscribe so you don't miss the next one.
New episodes every week that explain AI, cybersecurity, and digital risk in plain language for business owners. Subscribe so you don't miss the next one.
NEED HELP WITH THIS IN YOUR OWN BUSINESS?
If this episode left you wondering what your team is already pasting into public AI tools, you're not alone. Phoenix IT Advisors helps small and mid-sized businesses with IT support, cybersecurity, and technology strategy. Visit PhoenixITAdvisors.com and schedule a consult.
LINKS
Full episode: https://unhackmybusiness.com/episode/107
Phoenix IT Advisors: https://PhoenixITAdvisors.com
Craig Taylor's AI Policy Scan: app.aipolicyscan.com
Full episode: https://unhackmybusiness.com/episode/107
Phoenix IT Advisors: https://PhoenixITAdvisors.com
Craig Taylor's AI Policy Scan: app.aipolicyscan.com
Creators and Guests
Host
Bryan Lachapelle
Hi, I’m Bryan, and I’m the President of B4 Networks. I started working with technology since early childhood, and routinely took apart computers as early as age 13. I received my education in Computer Engineering Technology from Niagara College. Starting B4 Networks was always a dream for me, and this dream became true in 2004. I originally started B4 Networks to service the residential market but found that my true passion was in the commercial and industrial sectors where I could truly utilize my experience as a Network Administrator for a large Toronto based Marine Shipping company. My passion today is to ensure that each and every client receives top of the line services. My first love is for my wonderful family. I also enjoy the outdoors, camping, and helping others. I’m an active Canadian Forces Officer working with the 613 Fonthill Army Cadets as a member of their training staff.
Host
Mario Zaki
During my career, I have advised clients on effective – and cost-effective – approaches to developing infrastructure that fosters productivity and profitability. My work has provided me with a broad-based knowledge of business from the inside, with an expertise in areas that go beyond IT alone, ranging from strategic planning to cloud computing to workflow automation solutions.
