Screwing with AI on a core ultra 7 155U
I was getting bored with work so decided to fiddle with wsl Ubuntu and AI. I decided to go big and downloaded a 14B model in ollama. The office laptop has 32GB of ram. It’s waste because coding takes hardly 10-12GB. Anyway there was a proxy issue with wsl which I solved by importing certificates from windows. Then got ollama installed. Then got a 1.3B model installed for autocomplete. And then got the 9GB 13B parameters model. Running the 13B model on CPU was a pain. But the integration with continue worked decently. I ran into some code applying issues but nothing crazy. Only issue was slowness. Next I asked copilot how it can be better. It asked me a bunch of questions to figure out if the GPU is accessible in Linux. Turns out not. After a bunch of tries I reached a point where intels openVINO runtime was able to detect the Gpu. Then I asked the actual question that how much this gpu will help and the answer was 50% which is not really much. So I stopped. But it was r...