Skip to content

intelligence.md · Ep 01 · 8 October 2026 · 59 min

3 Ways to Run AI Yourself

with Ali Karpuzoglu · hosted by Kerstin Dallinger

The video loads from YouTube when you press play. Privacy

Watch on YouTubeListen on Spotify

About this episode

Ali Karpuzoglu, a software engineer with a master's in autonomous systems, joins host Kerstin Dallinger to explain how a business can run AI on its own terms. He walks through three options: using the computer you already own, renting GPUs by the hour, and buying dedicated hardware. Along the way he explains why self-hosting alone does not make a company GDPR or AI Act compliant, why small models suit repetitive tasks such as classification, and how to test a local model against a cloud setup you already use. He shows a small benchmark, a Hugging Face page that checks which models fit a laptop, and the hardware he runs at home. He then covers quantization, open weight versus open source models, license terms, and security and maintenance over time. The episode helps business owners, solopreneurs and small teams decide whether local AI is worth trying and how to start without a large investment.

Key takeaways

Why would a small business run AI locally instead of using ChatGPT or Claude?

Ali names cost uncertainty, rising hardware costs and not having to share private data as the main reasons (4:00). He adds that you may not want to pay for external API calls for small classification tasks if you do not need to. A model you host yourself behaves predictably, while he describes the OpenAI API randomly reacting differently even with the version pinned (8:24).

Does self-hosting AI make my business GDPR and AI Act compliant?

Ali says no. You still have to consider all the other flows of your data, and some fields such as education and legal AI have their own rules (5:35). He also says the model is only one part of the system, and without the right guardrails the model can break out (6:34).

Can I try a local AI model on my own laptop without installing anything?

Ali demonstrates chat.webllm.ai, which runs a seven billion parameter model fully in the browser on his Mac (9:53). He describes it as useful for experimenting and says he would not base a business on it (13:03). He adds that a laptop with 32 gigs of RAM is enough to trial some pretty cool stuff (14:30).

How much does it cost to rent a GPU for AI instead of buying one?

Ali shows RunPod, where you choose the amount of RAM and rent by the hour, and he thinks it is billed by the second (15:44). His example is an RTX Pro 6000 for two dollars an hour, while buying one would cost twenty thousand euros plus the rest of the computer (16:27). He recommends renting for short trials first and then calculating whether buying is worth it (17:45). The RunPod link is a referral link: if you sign up through it, Ninja Partners receives RunPod credits.

Is it worth buying my own AI hardware right now?

Ali calls this a very odd time, because prices keep going up, and he shows the DGX Spark price rising over the past year (20:41). He mentions budget options such as a used 3090 for around a thousand euros or AMD GPUs under two thousand each. He says the cost is definitely higher than using the API, partly because API prices are subsidised by investor money, so future prices are hard to predict (21:36).

How can I test whether a local model can replace the cloud AI I already use?

Ali suggests running the same requests through a local model alongside your existing cloud flow and observing over a few days how much the results overlap (26:21). For a classification task with known inputs and outputs, he suggests rerunning everything once a day on a rented GPU for about two dollars (28:48). If the model is unsure, he suggests escalating to a better model and then to a human, which still reduces cost (31:21).

Chapters

  1. 0:00Welcome and meet Ali Karpuzoglu
  2. 1:39Not your AI, not your business: 3 ways to self-host
  3. 3:37Why run AI locally at all?
  4. 5:11GDPR, AI Act and why self-hosting alone isn't enough
  5. 7:23Small models for repetitive tasks
  6. 9:13Option 1: Run AI on the computer you already own
  7. 15:06Option 2: Rent GPUs by the hour
  8. 20:07Option 3: Buy your own hardware, and what it costs
  9. 23:19Benchmark results: local models on real tasks
  10. 27:39How to test a local model against your current setup
  11. 33:07Which business tasks suit local models
  12. 35:22Why the harness matters as much as the model
  13. 37:14Demo: which models fit your laptop
  14. 40:38Quantization explained
  15. 42:19Open weight vs. open source, and licenses
  16. 45:19Running it long term: security and maintenance
  17. 49:14Q&A: hardware, NVIDIA alternatives, harness choice
  18. 57:07Wrap-up
All episodesJoin the next live session