DN #32: Open Weight Models, Airbnb for Compute & Why Nobody Works Less With AI (w/ Alexander Solod)

Alexander Solod:I haven't met anybody who's like working less with the like after the release of AI. If anything, everybody's like attacking on a lot more deadlines, like deliverables get a lot like more optimistic and deadlines get shorter.

Niklas:Hi, and a huge welcome to you, my lovely listener. So glad you're here. Today you're joining me for a chat with Alex. Alex is an AI developer, and I think we picked an awesome time to talk about it because we have so much happening in the area of open weight models. But before we dive into that, maybe a few words regarding yourself and your background. What led you into AI?

Alexander Solod:Yeah, sure. So thanks for having me on. I've mainly been I've been building in like generative AI more or less since the very beginning. I remember early on during like my college days I came across a paper around that was written like purely by GPT two and then at the very end we only found out like at the th there was a reveal saying, By the way, all this was like written above was written by a generative model and it wasn't written by like a individual and that completely blew my mind and I realized like whatever this technology is like I wanna be like a part of it. I wanna be there and this is gonna be like obviously very like transformative for everything. So like I jumped on GPT three as soon as I was able to get like a playground access and have been building with like AI ever since.

Niklas:I remember like back in the day when I was still in consulting, we did some predictions with like new ma normal neural nets, right? So we did some some small predictors for some some data like dwell time for containers and container yards and this kind of stuff. And then I stumbled across this little company. It was back in twenty eighteen, I think. Yeah. 2018, 2019, it must have been that has had written or that they had developed something clever. It was called J Gym Environment, and they made it to compare different I would say AI models and AI wasn't what it was today. The company who built this was OpenAI, and they got quite intrigued by them because they had published this little video where they show I've not sure if you've ever seen it. It's it's really interesting also for your listeners if you want to look it up. They kind of had this hypothesis of self-learning and they had like a very simple simulation of hide and seek where kind of agents were self-learning to play hide and seek in different environments and it was evolving. And this this was obviously this were kind of the I didn't know about GPT yet at that point in time. I hadn't seen anything. It must have been around the time when they started to really seriously look into that. And then I didn't follow OpenAI much, except that I learned that Elon was involved and they had very bright people. And the next time it popped up it was the launch of chat GPT, like in the first version. And I think you could already feel that if this improved more and more that this would would become pretty big. Since then a lot has happened. And I think currently the really, really interesting thing that we currently see is now OpenAI is pretty big. Anthropic seems to be even bigger in case of revenue. But what's also happening is we have this huge advantage advancement in Chinese open weight models. For me the latest one is QN three point eight. What do you think? Have you played around with open weight models a lot? What is what are your thoughts on them?

Alexander Solod:Yeah, quite a bit. So I have like Gemma Four currently running on Mac Studio two, which I just use for general like per personal agent tasks. And I do think the future of like AI models is gonna be like somewhat similar to open await or something like on on device models that people just use for for general purposes and the current like big soda like state s state of the art large models are gonna be like reserved for some of the very just either enterprise tasks or like those very like edge case computationally or intellectually heavy tasks.

Niklas:Yeah, so so I think there will always be place for the smartest models, right? And and I also think we don't really we have no idea about the limits. We I also me I often make the mistake of benchmarking gu against us humans in general or relying too much on benchmarks. When I when I think about the status quo, I think we have seriously no idea what the limit on intelligence is, and I'm one hundred percent sure that some some people say that the limit of intelligence is the energy of the universe, kind of. I can follow where this idea comes from. I and I think that these will always have their place. But on the other hand, there's a huge advantage if I can have like a small model on my phone. I don't need an internet connection. I don't need a subscription. I can just run it by myself, right? So so what will will this mean in your view?

Alexander Solod:Also to go back on the point around benchmarks, like okay, I I wanna say like even like hu human performance isn't really I would say a good benchmark in and of itself, just because performance could be so so varied. Like if you think about like the average like individual Your local open weight model or definitely the models that we have like right now through like the big players like OpenAI and Anthropic are definitely better than like your average individual at using a computer. Does that mean we reached AGI yet? Does that mean like they're like outperform they're outperforming, let's say, like a good fifty percent of the population? Th w what does that mean? Does that mean that we're in a good like in a good position where like we can say like AI is better than like mo most people? D d does that mean anything? Is that a good benchmark?

Niklas:Yeah, I'm kind of stuck on that. I seem f for average so w I think what is quite clear is we have this new technology that can do things quite well that no other technology before was able to do yet, right? And it it also seems to be good at Fine I I have to be a bit careful, but it within like huge amount of data it seems to be very good at finding connections or similarities. So what I I feel these models are generally quite good if you think about them as a representation of a really large amount of knowledge. I'm still still unsure how good they really are with things that are very little research. Like how how far what's your take on that actually? I've never thought about it as well. But if you think about like reasoning up from like physics or something like that, how good are they at that actually?

Alexander Solod:think w I'm I'm trying to remember, was there a paper published or like some somebody wrote about like the fact that they would never be like if they were given only the information up to Einstein's time, LMs would never be able to like make the jump and come up with a theory of relativity or make the the connection that or the discovery that Einstein did. So they aren't great at synthesizing like new novel advancements. However what if you take a look about them conceptually, like what they are, it's more or less like a amalgamation or compression of like a whole bunch of knowledge over a period of time. So they're very good at trying to find like the the connections and trying to find trends and like patterns in the data or in the things that they're working on. So we shouldn't be like expecting them to make any sort of like wild novel jumps outside of like what is currently there or outside like the reasoning that they present. Like they're not gonna make like some revolutionary like a brand brand brand new revolutionary like paradigm shifting change, but they might be able to discover things that we overlooked or okay make a novel connection in like currently existing research that like we like as humans just don't have the time or maybe even the mental bandwidth to to connect and to realize.

Niklas:And what they definitely do is they give us a new UX. So the it's just so nice to interact like with another human being. Like good chat is just such a normal way, especially with speech, to interact, right? And they are just very good in tr in translating this into machine commands as well. So to use your computer, probably better than most humans, as you already said, and just translating your text messages into action is is they are really good at that actually.

Alexander Solod:Yeah. Unlike a broader like I would say more like philosophical level I try to view like LLMs as like doing two things. I view it as like a but as a filter. So we have a whole bunch of information now like on like on the internet with with all the books and like available constantly at our fingertips and we don't have the time or the bandwidth to actually read to go through it and like to to maybe like identify all the important all the important points that we like might want to like read or look at. And these like LLMs do a great job at trying to like filter out and point out the like information that we would might want to extract from them. And then I would also like to think of them as like a like an enzyme or something to like shorten the like amount of energy it takes to complete an action. So for example for this like let's say take app development or like prototyping. Before things that would used to take weeks previously would now take you hours or like yeah and and sometimes minutes. So the amount of like you could now process a lot more information and get get a lot more more things done and also the amount of effort that it takes to go out and to get like to a like a v0 or v1 of something is significantly like lower than it was previously.

Niklas:And beyond that, I think we we are still to unlock one huge step, and that is still on device AI. I think Apple is gambling on it, kind of. They they just sat back. They I mean, they already knew, at least in my opinion, that something was coming. Otherwise they wouldn't have built all these neural engines into the one laptops and kind of prepared for a future where these things are used. more right. They still made a conscious decision, I would say, not to heavily invest and compete with like the Frontier Labs or with Google on it, but they kind of decided to wait. Now with what's happening actually, especially with Q and three point eight, we might see on device models quite soon that are fairly capable. What what do you think about that?

Alexander Solod:Completely c completely agree. I think it was Google earlier this year that published or that released that they're trying to bake model weights into a particular chip and then just have a single like model just completely like all all the weights like saved into like a single a a single chip. So you have that chip to like offer it's a single model that'll like offer all the computing needs. And I think that again, like as these models grow more capable, like we would no longer like we we would be able to deal with the possibility of having just like a dedicated like like neural engine come with a like a particular device kind of like a graphics card and then have all the like information like processing be done specifically through o only through that chip.

Niklas:Yeah, I think th that that's obviously one way to go, probably the most efficient one. Another one that I was wondering actually today about was now that that we are seeing like smaller models getting really good and demand for tokens will further increase where we see something like decentralized compute on like consumer hardware at home. Like you say you have your Mac Studio, you could also rent it out in idle time, right? So who's gonna build that maybe it already di exists. I didn't take a look at it, but I would be really interested in who is going to build this decentralized compute chain that runs small models and sells tokens.

Alexander Solod:For sure. Like there that's definitely another possibility. where it's like kind of like the like bit BitTorrent network is the first thing that comes to mind, but for for compute where a bunch of people like you could opt in or rent out or like your at the Airbnb for compute, where you rent out your spare like anything that you have laying around and people can use it to either for inference or for whatever else. I know there's a couple of attempts going at this right now, but we still have not like reached the mass like level of adoption for it to truly be like vi viol.

Niklas:Yeah, I saw it in the like in the crypto space because this is one of the more obvious early adopters and I think I saw it in in something like there's a very small chain that that whose idea it was to store data forever. Like they it's called R weave and they thought about this compute layer quite early. But now I'm really wondering if something like this is feasible. Maybe somebody else will do it. But If the models get small enough and demand for tokens is high, why not rent out your hardware at home? Because it it it seems quite quite obvious, especially if you have solar at home and you can offset your energy costs, maybe decentralized compute will become a huge topic.

Alexander Solod:For sure. I think it's possible to I think like for like your most consumer use cases, it'll be relegated to either like a small like on device model or a dedicated like model chip running your like basic operations. And that and I think like we still haven't really cracked like the right user interface for AI just yet. Like chat it was obviously a good V1, but I d I don't think the future of AI really rests in everything being a chat interface.

Niklas:Yeah, I I think it depends on what you call chat. I'm quite sure that the best way of input is voice. And the best way of output is some visual. So either text or some other kind of visual which is generated. So for us humans, obviously we can transfer it's easiest for us to transfer information if we speak currently. So I'm quite sure that voice input will will be very large. For the output I'm uncertain how whether it will really stay text or whether we see something else or it will just be more like what we are becoming like generated graphics or something. What do you think about that side?

Alexander Solod:So I think like yeah, i i i it could go in like both ways. I do think the future of like kind of having some element of like generative UI, where you go ahead like you speak and then like it can either like g generate a new interface for you or it like navigates to a particular like cr cr pre built but critically existing interface. But as for yeah, like it's we we we we we we really really just haven't really figured figured it out for like w what the best way to interact with it. My one like issue with voice being like the end all be all like for everything is there's just some operations that like You y you don't always like constantly want to be speaking to your phone like tw twenty four seven. You would still want to have that like the the ability just like to like let's say like I don't know, highlight circle something and send it send it off to an agent. Or to be able to interact with it like in a non-voice way.

Niklas:Yeah, so so I'm kind of sure that the end will be some brain interface, like where Elon Musk is going within Neuralinks. So so I think we'll go beyond it. Currently the way like typing on a on a keyboard is just really inefficient. Like it's not a good way to communicate with machines. Voice is a lot faster. processing wise, it's f for us, it's faster to process with EIs. So some kind of reading. Obviously if you can di connect directly that that is a lot faster than everything I've said before. that's for me at least quite far away, but I'm also not like in this in the Elon Musk world. he might have this big vision already in his mind.

Alexander Solod:Yeah, for sure. I think like we're still very, very far from brain brain interfaces. And then even if we're not far technologically, I think like the adoption of brain interfaces will still take us quite quite some time for people to become like useful or people to become accustomed with that modality.

Niklas:Where I think AI will make a huge impact is in predictive health care. And this

Alexander Solod:Mm.

Niklas:seems to be a a field also where I seeing huge development. I think obviously r computer vision is a huge topic, so probably radiology, a lot of things will happen there, but also in like just gathering the data and doing predictive health care. What do you think are the biggest trends in that space?

Alexander Solod:Absolutely. Like ra radiology like it it's funny. Radiology has been like the field that like has been like most talked about with automation, ever since like I would say like the mid the mid twenty tens like it probably even earlier with computer vision and people saying like radiologists are gonna be out of a job and here we are in twenty twenty six and still a booming field, so It's not not going anywhere. But I do think AI finally gives us the ability to basically ingest like the large amount of like patient data and patient information that we like no like individual clinician ever had like the time nor mental bandwidth to like accept and to provide s people with like truly individualized and like personalized care.

Niklas:Ye so first of all this radiology topic aside, I also don't think that the people will be out of their jobs soon. I think you will for f at least for a long time you will always have the want to have the quality control on these really huge decisions that that you're making. And like a surgery or something like that, it's it's really big. Like you getting an indication from from computer vision is a very different story than having like the real suggestion on a very serious surgery or something like that made by a computer, right? So I think assistance is the the way there. Then again I feel like if you if you for example can by sampling whatever it is, your poop or something like that, make see diseases really early. And this is predictable and relatively cheap to do, I think this will be one of the spaces where Where things will be moving really fast because you can get away with less doctor visits if it's reliable enough. You gather a lot more data over a long time here horizon, so you see changes actually. where do you feel like do do you have an idea what is currently like in this predictive space the most relevant that's ongoing?

Alexander Solod:In in the predictive space, not too sure, but I can talk about just like broadly like the the kind of work that is being done in like medical AI. And that is healthcare eye. That is basically there's a lot of like processes and stuff that clinicians deal with and also it's like deal with in the the back office that like many people don't don't know about and they take up a lot of like time, where it'd be from like re reviewing patient notes, writing like the actual like meeting summaries, submitting the like notes for billing, there's a whole lot of paperwork and now we're finally that that that is outside of the entire field of just practicing medicine and providing good patient care. And a lot of work is currently being done around trying to make sure and like trying to get clinicians da back to doing what they like doing the most, which is actually being with the patients and like providing them with listening about them and helping them like provide like the best care that they actually can. So I think like w with with AI, like it is the like w one of the best things to happen when it comes to like medicine because now like we y you're capable of ingesting so much more of like information and about the patient about like what is currently like what what they're currently dealing with and some things that you might have like previously not had the bandwidth or the the ability to do so.

Niklas:When you see AI deployed in healthcare currently, what do you think are the biggest challenges at hand?

Alexander Solod:W for the first one it's like obviously like regulatory and like safety. So are we going ahead and like is it actually doing the thing that it's supposed to be doing? is it are there like any things that we might potentially be missing around are are there any like consequences or stuff like we we we not we might not be aware of? Are there any like model biases? the second one it's like obviously like do we do we trust the system? do we do we trust it to be making the decisions? Is it making like the the right ones? And then like third, like on a more like infrastructure level, like is it like secure? Is it processing all the information like the r the right way? Are we going ahead and are are we going ahead like are we treating the like the patient information and like not violating any of the healthcare, like da data privacy laws, all of those are like I would say like the biggest questions or th the biggest things to keep in mind.

Niklas:Yeah, and and I think that's fine because in healthcare you want to make sure that the people are actually better off using something than not using something. That I think that should be should be quite clear because the impacts are high. then again there are probably a number of early wins now where AI is used to improve things. Do you have some examples?

Alexander Solod:Early Windsor AIs used to Yeah, so it's like I think the best example of like where like AI completely revolutionizes was like with ambient AI like AI scribes, where before we would have like it would be like a first or second year college student going in, like following a doctor and taking down notes for whatever like w whatever it happened during like a patient visit. And then now we have just like ambient like AI systems that can go ahead and like do do all of this and like do all the documentation and note taking for like for for clinicians. It's b it's been around for over years but think that that's like the number one like easiest like highest like e easiest win that we've had like so f so far.

Niklas:I I've heard about this actually for veterinarians. So the people that deal with animals as well, they they tend to go from farm to farm and sit in the car a lot. And this just having proper

Alexander Solod:Mm-hmm.

Niklas:dictation in the car where they were able to dictate into it what they just did on the visit is a huge time saver for them because they can obviously do it while driving. So I f I also find this case interesting. Then we see a lot I think I'm not sure it's at the edge of healthcare, but people now taking the data from their Whoop or something like that and doing deeper analytics on it. It's quite big on on X currently. what do you think about that topic?

Alexander Solod:Yeah, I think that's kind of like the next level of like where where we see like individualized and like personalized healthcare going. Where I again like you see your clinician like once every year or so and most of the times like the they they they take a blood test and it's just done to make sure like, i is there anything different? Do we c are are we catching anything? meanwhile th now with like with the with the with the whoops and then like with like proactive like monitoring of like your your health records, you're able to just see in real time and then just a whole lot more of information around like okay, well what is what what is happening here and try to catch changes in real time and try to be more like proactive and preventative with your healthcare. And before like if previously like it was like you're able to do this but only like if you can see like a clinician, like your the Brian Johnson of the world I'd see a clinician every like week or so. and try to like d and do regular tests. Now it's kinda like democratizing and opening this up to a like a much broader like area of people who could now like go ahead and take much better track and much better records of their of of their health.

Niklas:And beyond that, I think at least for me, I sometimes forget how small of a percentage of the world really has access to first class healthcare. Like it's limited

Alexander Solod:Mm-hmm.

Niklas:to to a few hundred million people, but a large amount of of the billions of people living on this planet still lives in a world where healthcare is second grade. So probably just being able to bring in these analytics more long term and have cheaper health care will will move the needle a lot for these people who have very little access today. So this is I think is also a very interesting trend. If you look at the

Alexander Solod:Absolutely, yeah.

Niklas:near term impact for AI and healthcare, what do you think will be the large topics over the next one to three years?

Alexander Solod:So the biggest impact I think like yeah, the democratization of like healthcare, the like speeding up of certain clinical processes and like trying to do like the speeding up certain clinical processes and like the increased quality of like some clinical healthcare as well.

Niklas:We have another trend and I think especially in in these episodes where I talk about AI, I also like to talk at least quickly about the the impact that most people fear the most and that is the job loss side of of AI. What's your take on on that side and what would you do as a jung young engineer today if you started up?

Alexander Solod:I think it it's an interesting question. First of all, like I I'm sure like you you've seen of this, but like I haven't met anybody who's like working less with the like after the release of AI. If anything, everybody's like p attacking on a lot more deadlines, like deliverables get a lot like more optimistic and like dead deadlines get shorter. But I think the most important thing that you can do right now, like in the age of AI, is like to try to get yourself in a position that like has actual stakes and where you're making some sort of decisions because as the Age old saying goes like a machine can never be like held responsible for his actions. There would like always be somebody like the people that people would want like at the helm and somebody they could look to and say like who who made this decision. So if you're like as as long as you're in a position where like you you need some sort of like you you're making some sort of judgment, whether it be like in product direction, engineering choices, like whether or not like what to put on, I would say your position is more or less like safe.

Niklas:I've been thinking in the same way for a long time. I always thought more about intelligence augmentation than actually artificial intelligence. So a lot more about people just being able to do a lot more. And I think that's this is what you said. It's kind of the paradox, right? Now people

Alexander Solod:Mm-hmm.

Niklas:are less dependent on other people, so there's less waiting time. So they can actually be more productive and do more and they tend to work more. I think it's it's it's a weird trend. I'm And I'm sometimes questioning whether these big company layoffs are just blamed on AI for now, or whether it's really the AI taking the jobs. Like w I'm not one hundred percent sure which side it really is.

Alexander Solod:Think they were blamed like on AI because over like the early pandemic and stuff companies like of greatly overhired and so this provided them with a very like convenient excuse to say like we're laying off because of this like amazing wonderful technology and not because we accidentally like hired like a hundred more engineers like during the boom than we needed to. So it's able to like work as A advertising and B like in a way to cover themselves for like a mistake that they made earlier. But I think it's kind of also like going back to bite them because I'm not sure if you take a look, but like the public sentiment like around AI outside of like I would say the small like SF bubble sphere is incredibly, incredibly, incredibly negative. And I think a good part of that is because they see like these as AI layoffs and they think like, AI is coming, like AI is coming for our jobs. Like we're we're gonna be jobless like that, that that's it.

Niklas:Yeah. You have both sides. I think this one, there's a lot of fear in the space and it's quite obvious, right? Because it's a huge amount of change, people are not in control of it, and it might threaten their livelihood. So obviously people are scared. And I'm also like if that the things that people like Elon predict at times really become true, I I think it's at least partly justified, right? Because there will would be the question what everybody does. then again I I would agree with what you said first. So there seems to be this new metric in town, which is kind of revenue per employee that companies get graded on. And the more revenue you're actually doing per employee now matters a lot more. And that's just a very different story to a time before where kind of head count was a justification for your valuation. So if you were a big company. And you wanted to have a high valuation, you had to employ a lot of people. And I think that mindset has just changed a lot. Actually, before AI, when Elon came in and bought Twitter and let go of so many people and it was still running, I think that was the biggest change actually in the story. When you think about like preparing oneself for the future, like future with AI, what should people do to be less afraid?

Alexander Solod:say like the number one thing is just work with the models. I've been speaking to so many people who like their most of their experience with AI has just been using like the ChatGPT like chat bot and they haven't like ventured out into Codex or Cloud Cowork or any of the like more proactive instances of of of AI. I think that like g going back like December of last year of twenty twenty five was like the really big turning point where we went from these like reactive like with the release of OpenClaw actually I I think that was like the big shift in the in the meta where we went from these like reactive like chat interfaces where AI can like only do what you describe it to to it now going ahead and like being able to proactively like suggest tasks, do things on a loop, like re reach out for you and do all of this like cool do do do all these things like that it previously like w was not able to do without your assistance. And many people are still stuck in that like first phase of like AI only, like is is somebody that that that will speak when spoken to, that it can't be off like doing like work for you in the background. And so I think that's like the very like when it comes to like AI proofing or like learning, like really like be becoming acquitted like with the technology, that's like the step step one. It's like actually use the full capabilities that are on offer and like become like become familiar with them, find out where they help you, find out where they where they

Niklas:For yourself, if you look at your career advice and you could give something on to other people at your point in time, what what would you say was the best piece of advice that you ever received?

Alexander Solod:Mainly like I would say just like keep Keep momentum, I think, is the very like the the the best piece of advice for like that I ever see like both like professionally and like personally where if you're like pi picking up with something and you're like trying to like i e either learning a new task or like you're trying like you you're seeing some success in a career, just keep keep going at it and try like tr try not to slow down and try like start start picking things up because like it's it's it's a lot easier to keep going than it is to like to like slow down to like begin and like s start from start from the beginning.

Niklas:Yeah, I totally agree with this. If listeners want to follow you, where can they find you?

Alexander Solod:you can find me on Twitter, GPT underscore Alex. that's the one where I'm most active on right right now.

Niklas:And any final hot take or prediction you want to leave people with.

Alexander Solod:mainly I would say like the like biggest the hot take is again like we we have still not fully figured out like the best like y interface, like the the the best way to like interact and interface with with AI. So I would say like that entire thing is still like completely like up up and going. We have people building out like agentic subnets for like having agents like navigate the net and like buy buy things for us. We have people trying to do agenti commerce, but I think that field is still like as green and as brand new, as like the definition of agent was like two years ago back in twenty twenty four when everybody was building agents but not no two people really had the same definition for what an agent was. So I think like if you're trying to get into the field then how do we interface with agents and how do like agents actually like go out and do things better for us is the best way to go to to go about it.

Niklas:Thank you, Alex. It was an awesome episode. And to you, my lovely listeners, see you next time.

Alexander Solod:Thank you for having me on. Bye.