AI Assitant Computing

Why do we buy personal computers? Do we get one because we need it for school, work or gaming? Do we get it to help figure out our finances, write emails and surf the web? That’s all fine and good, but if you step back and think about it, what are we really trying to do all that for? We surf the web to get information, to be entertained or to communicate. But when we step away from that computer, it no longer helps us.

Of course, we have our phones now which have become our personal computers, and we use them to do very similar things. We also have smart home voice assistants like Google Home or Alexa. But these mainstream tools hit a frustrating wall: they simply don't know enough about us. If I ask Alexa who installed my solar panels, it draws a blank. I am left digging through old emails and paper clutter because standard assistants are disconnected from our actual lives.

I propose that we can do more. If we expand the role of personal computing to that of a universal helper who actually knows our context, who can help us with everything no matter where we are to get real things done, then that makes a personal computer immensely powerful.

There are services emerging that offer just that. Facebook’s Muse for example connects to your personal data and proactively summarizes, advises and does things for you. But that is a canned experience. It’s time we get back to what a personal computer is really good at—unleashing your personal productivity completely customized to you with the full power of a real computer.

For the longest time, I’ve been on a quest to untether myself from my desk and laptop so that I can be anywhere and still access the full context and power of my computer. I wanted to be able to go for a walk in the park and continue work as if I were sitting right at my desk. And I think I’ve cracked it.

Imagine being able to open your phone, tablet, or computer and just ask it for information or ask it to do something, and it actually figures it out. Recently, while dealing with a frustrating gutter contractor, my setup kept detailed notes and guided me on how to handle the communication until the job was finally done. When I was shopping for a new couch, it already knew the exact layout and dimensions of my living room. It even completely reorganized my household financial accounting using double-entry bookkeeping: I simply asked it to do the job, fed it my PDF statements, and it read them and imported the transactions without missing a beat. All of my personal data stayed secure on my machine, and now I can ask complex financial questions that weave seamlessly into other household decisions.

That kind of freedom and capability sounds like science fiction, right? Well, I’ve done it, and you can too. Inspired by what's possible when we expand our minds beyond passive tools, I set out to build a truly personal system. With the tooling available to us, it was easier than I thought, though it requires some advanced knowledge. Fortunately, I’ve done all the work and will be making it public. Long story short, using a combination of a custom harness, opencode, git and a mobile friendly front end like Trello, you too can setup a workflow that unleashes the power of your computer into your own hands while keeping all the data private (with the caveat that if you don’t have the computing power, you will have to use privacy-friendly public AI model hosting).