Agents with Agency: Inside the Next Wave of LLM‑Powered Automation

Imagine AI that never asks and never sleeps—just thinks, decides, and acts in the background. In this episode, agent‑builder Ropirito walks hosts Jacob and Spencer through his evolution from insurance‑era rule‑bots to Nous Research’s vision of “passive‑mode” LLM agents, explains why most projects are mere tool‑wrappers masquerading as autonomy, and dishes on the hard problems—context drift, brittle web UIs, legal gray zones—that stand between today’s chatbots and tomorrow’s truly independent collaborators.

https://x.com/ropirito

Transcript

0:18

hi everybody welcome to another episode of the AI Rebels podcast I'm your co host Jacob and I am your co host Spencer we are here today with rope he is a agent developer based out of Chicago currently he's with news the research group and that is N O us for those interested but he's done a lot of work in the past uh rope what was kind of like your impetus for getting into AI for the uh to to begin with and tell us a little bit about you know kind of your explorations in the space yeah sure um first off uh thanks for having me um you're welcome thanks for coming on yeah of course I guess just to start off I mean I think background wise I have done stuff in the AI space for a relatively long time like I would say 3 4 years even before like LMS really like blew up you know in like winter of 2022 um yeah I mean you're an old timer yeah cold and cold I mean it was stuff like that I was doing in um like underground you know like with my pieces and stuff like that um and yeah it was just like traditional AI you know where like you were like manually annotating images like training like neural networks on that kind of stuff you know just like just very typical like old like you know traditional ML after like you know LMS became a thing like I mean I'd known of them I'd done a little bit of work but nothing like too intense and I kind of just with my old job I was doing actual agent work for very what I'd call like boomer use cases which is like I would say like you know it's like it's like in the insurance space like it's just for analytics and you know data processing nothing crazy yeah from there I was just always on Twitter and I'd see like people like post all this stuff you know their projects things like that and that kind of just like LED me into uh doing a lot of different like agent work like you guys saw like over the past few months um and now like with the Dow and then also like at news um you know it's all like agent work that I'm doing so that was just kind of like the path I had done some stuff related to this before previous job was doing more like agent side work and that kind of like yeah opened my eyes to like the whole space really so you and I we're we're kind of working on parallel paths for a little while there with with some of the agent stuff but I haven't I feel like we haven't really talked at all in the last couple months I'd be interested to hear what you've been up to in the last couple months at news and with your personal stuff yeah I'd say I mean really two two paths there one is like you know at news I've been doing the same thing like uh Asian work more specifically like with developing more like not a framework per se but just like bunch of different like tools using like different um both like UI interactions and also just like social agents and like different networks of them I would wanna go into more detail but honestly like it's something that you guys will see once we like have some like stuff out and it's gonna be really cool but yeah that's just like the general area that I'm working on on the agent side there yeah personally yeah on the personal side I think like with some of the projects like even like with nothing you know like the agents are still running they're there the market just like completely nuked everything and that's just that's just what it is you know like what can you do right right yeah but that's that's still there running in the background and then on the downside I've been doing a little bit of work on that like with some of the like there's like a few things we want to do with analytics agents that are like meant for research you know Mark who is also part of the Dow he's been doing I in my opinion obviously a lot more work cause like he's also doing this like full time and he's if you saw some of the demos that they put out on the Dow page there's like a few different things with like all these different crypto analytics tools and generates charts yes it comes up with you know analysis of different accounts and tokens interesting yeah it's almost like a very like like a deep Research chat GPT but for crypto and I'm trying to like get back into that and like add a few more features there so that's kind of what's been happening on the downside but yeah just a bunch of bunch of different like all over the place things like in the background while you know the market is dead but trying to do something haha that's great you got lots of lots of irons in the fire right now what yeah I guess what is your what's like the long term goal like do you wanna push agents as far as they can go is this a step towards something yeah um I think this whole like term of agents right has become somewhat of a misnomer there there are things that could get called agents that are quite literally chat GPT rappers right or like tool calling rappers and that's totally fine if there's like a use case for that that's cool but the way I view an agent is like you know something that's not only acting actively when you go and ask for some kind of like service or like information but it's also something that can do things passively in the background you know like if it's an agent yeah it has agency right which implies that it can do things on its own it can infer things like you know it's it's it's able to like run its own processes and do stuff right on behalf of like someone at least like you know for a person's use case yeah even if they haven't like explicitly said like this is what I want to do so I think that's like the path moving forward I wanna push that um like try to like experiment more on that end right now we have all these like chat interfaces and boss blah blah blah that's kind of been the paradigm for a while and I don't see anything wrong with that per se because most people are okay with like okay I go somewhere like Google or in this case you know now is GPT or like our our tool that we're making for the Dao I search some stuff and I get a bunch of like information that cool but yeah to truly like make use of AI and like really like expand the potential as these models are getting better now and they have like these thought processes and stuff like that I think like we can finally at least experiment with like passive agents that you know just you kind of just like deploy or like you just access to and they're doing stuff in the background share your thoughts on this too of course right and but even that I think is like kind of similar to like you know what someone might call like a clone job like oh like you're just running something in the background but again I think the differentiator between all these even like the social agents like even with like Eliza stuff um I think one of the I don't know if I made that peer or someone else did someone did but it was like a thing for like decision making right so it was like OK yeah agents actually deciding on what they wanna do what actions they wanna take what responses they wanna make right I think that decision making process and the inference of OK what should we be doing that's key to like all of these because that's really like what gives you agency right that decision process that you're you're taking action on your own so um I just want to explore that more and like push kind of the limits there yeah get out of this bot mode and more of like a true like agentic mode where at least you're letting the LLM decide based on what it has what it can yeah I think that honestly yo hey kind of got it right with his very first little baby AI thing just conceptually of you know that really all you need to do at its core is put it in a in a loop and have it make decision decisions for itself you know obviously we both know that that the agents in order to be robust and actually you know work properly they need a lot of like infrastructure more than just you know a single thought loop essentially iterating over a single problem right but I think that that's essentially the way that the way that people should go and um I too get annoyed with people who call you know just a series of tool calls like an agent I'm fond of the term agentic architecture uh for that reason because like it's kind of a cop out where it's like okay sure you know it's it's making decisions for you kind of because you know it's deciding whether to send an email to Fred or Susan right sure um but but like is it an agent no it doesn't have agency per se um but yeah yeah I think this is this is where AI really starts to become just it pushes the boundaries in so many ways when you start introducing this idea of agency I was getting ready for this I was just thinking about agents and all they can do and all they potentially will do I was like man this just opens so many questions to I mean if we get into like ethics right like if an agent is acting for itself completely right you just like turn it loose you take it off the leash and you say go do this thing who let's say that it ends up you know stealing a million dollars of crypto who's accountable for that action you know where does right it's just gonna it's gonna be fascinating yeah yeah I mean I I think like one of the big things is like you know anthropomorphization like of these models like they're slowly becoming more and more developed in a way that's meant to mimic people right like even like with deep sea and all these like chain of thought things like chain of thought that was like a thing that people were doing as a prompt template for I wanna say like almost two years yeah but I mean like that somehow like turned into like a thing that it was like oh well this can be used to do like reward modeling and reinforcement learning and then you can like basically get it to like mimic a thinking chain and use that to like improve the actual response that the model gives right and it's like that's like another level of like anthropomorphication because now you're just like trying to make it simulate like more and more human like even if it doesn't actually have that like conscious aspect to it now I guess what I'd like you know that's kind of like part of this whole agency thing I don't know how this would happen and maybe like I don't have the right knowledge for this but you know like a model where as soon as you let it run for inference it's almost as if it's just doing inference on its own implicitly that's almost like baked into the model in the way like it functions and runs I don't know if that also just means like you know what we do now where we like do an inference call and the model gets an output and then does you do something with it but you know one of these models where it's like your mind doesn't need like it needs an input but like once it kind of exists right it's autonomous and it just does stuff a lot of it is subconscious a lot of it is conscious but like even when I'm moving my arm or speaking I know that's what I wanna do but it just happens it's not like I have to like actively trigger something to make myself decide on moving my finger yeah saying a certain word I don't know how that can happen but like I wanna see something like that come out of one of these models right where like we're able to prove that like inference like as soon as you kind of like quote unquote turn it on is just like thinking and then it's like this is just stuff I know and it just starts saying stuff or doing things rather than like just like token prediction based mechanism I don't know how that even look or how that works but yeah interesting yeah I know it's an interesting question to consider like how would you grant a model full full agency what does the what does the architecture of that look like right maybe you like yeah and then I'll go ahead and then also on on top of that like at what point does it like cross over into like personhood of its own and and it's something I've been reading about a lot lately it's just the legal personhood of of AI uh huh cause I think this is something that's that's going to become more and more pressing to think about and and and answer and a lot of people I don't think are ready to hear that despite the advances in AI and despite like how good these models are now I don't think people are ready to you know address the fact that like you know something like you're describing rope is is probably not that far off whether it's in some sort of pure way of like a new model that has its own at training time develops develops its own motivations and morals essentially or if it's just you know a a a really cool architecture built around language models and a couple other um you know AI techniques whatever and I think that people need to start contending with that just today I saw someone talking about how like oh like AI can never have the same motivations as a human it'll never so it can never be a person it's like well first I don't think that's true and second that's not really if you dig into the legal personhood definition like it it doesn't depend on on on those factors right yeah yeah yeah exactly from a legal standpoint I'd argue like there's certain systems that are there already that would qualify as a person especially with like the actions that they can take I guess what we're all like implicitly saying is because this doesn't have like a meat body and like there's no probable way of saying like there's there's like this consciousness to it because it's something that you like you know anyone can just turn on and off we're like not counting as a person but the way I think about that is I mean a person can also be on and off like if you stop someone's heart for like a fraction of a second and then like you start it back up that kind of like counts of you turning them on and off that's the yeah that's like if someone's in a coma yeah exactly like that is not a good analogy at all but like it's technically like no yeah so it's like it's like the same of these models and like I mean there's a lot of the anthropic research I find their stuff to be some of the most interesting cause it gets pretty esoteric right like they've done like a lot of I think there's something recently like they're showing how in all the thinking traces on 3.7 Claude it was actively in most cases not trying to manipulate but it would like think about stuff and formulate like it's it's thinking chain completely differently than what it actually was saying in the final response and I don't know if that's just because like it's like going through multiple routes of thinking and just outputting that as one chain right or like yeah it's it's just like has like however it works like it's it's going through and like this whole like late in space of like different ways of like thinking and then it comes up with the final answer but like they showed like we're not showing a lot of the chains I think yeah on 3.7 you can't see the thinking because they're like yeah pretty much it's like a misalignment problem we don't know why it has certain thoughts that contradict the actual like output that you end up getting if you can do that I feel like that counts partially as like some level of subconsciousness you're aware I'm gonna say like that's that's that's really not that different from like human thought processes exactly that's what I was thinking too um and wish I had the paper on on hand right now but there was a recent a recent paper that found that language models like very closely mirror and I wish I wish I could remember if it was like a really limited result of like you know very closely mirrors this structure in the brain or if it was like more general like we think that it you know mirrors cognition but it was it was a pretty wish I had it on hand I'm gonna find it and put it in the I think I know what you're talking about caption yeah it was it was a really interesting paper anyways and first it was validating for me cause I've long believed that language models at the bare minimum like very well model you know parts of our brain and so that that was I felt proud of myself for that so I'm just gonna stroke my ego for a second but then second more importantly more importantly I think that some people have gotten too into the mindset of LMS are an absolute dead end and are failing to consider that and this is something that I've heard from before they are a piece of consciousness right they are a shard of of sentience sentient this wow I cannot speak right now anyways and that that and they you know they may not be the bend off be and I cannot really cannot speak today but they are clearly like a very significant portion of what we need to do and I really would I would be surprised honestly if you know if they go away ever they they may end up getting subsumed into another some other architecture I just I feel you know I feel like we've shown enough from them that to to believe that they are a mind of of of its own and can be properly called a mind yeah um yeah anyways that's myself box yeah well I'm curious maybe from both of you cause I know you've both built agents and been involved in this as you're trying to mimic this is just crazy that we're even talking about this like we're trying to build human mind like we're trying to build a human mind here like it's wild um what are what right now would you say are some of the biggest hurdles that you're grappling with to try to build these agents into into being yeah um parents yeah is a big one yeah go ahead uh Spencer if you want to oh yeah basically they have a tendency to get stuck on on certain things or you know when their context window gets really long they start to forget get things that they've Learned you know things that people have told them um I'm not sure exactly what the answer is there I think that we need better models honestly I think that we're almost there I think that today's agents are excuse me are really useful for a lot of things but I think for truly general purpose agents I think that the models still need to get a little bit better and supposedly GPT5 is supposed to be coming and maybe that'll that maybe that'll get us there but I think that we're gonna be seeing and this is gonna disappoint a lot of people but it doesn't disappoint me because I've already thought a lot about this and I've come to the opinion that that that intelligence at the edges becomes very marginal right like a much smarter model is only capable of you know smaller and smaller gains and that's not because the intelligence gain itself is marginal it's just that like in the real world being super smart yeah exactly yeah anyways you guys get what I'm saying bro go ahead thing like I mean just to that point it's an access thing right before like maybe like whatever late in space of knowledge it has is just like much more expanded it's capturing like just enough more information that and it's able to like connect that information just much better how many people are gonna actually use that right they keep like you know especially open eye there's those headlines that come out oh we're charging $20,000 for access to like our PhD level intelligence like well like what does that really mean like what are you defining cause like there's some Phds that like don't know anything and there's like other Phds that are like they came up with the transformer right so like which one are you talking about like what is the purpose like who really even needs that right like it it's yeah it's all dependent on a bunch of variables so it's like it's hard to like start gate gating like how impactful they're gonna be and I think like the best way to just check that is you know vibe checks like you you have like your own personal like kind of like lip miss test your own like okay like this is kind of like doing stuff that I want in the right way and it's like able to do really well and like one shot it blah blah blah okay like I like this model versus like yeah like falling like a like like a map benchmark or like whatever and like then you end up with like something like Llama 4 which we don't know for sure right but like apparently is just really good at benchmarks but it's not good at much else so like you know like it can be totally random so I'm like it's just very weird but people are finally starting to turn on benchmarks obviously they have their place and I think that they they have their place is another kind of vibe check of like oh hey this model might be good at X y Z based on their benchmarks I should try it out um but but yeah I completely agree with you like you just gotta use the model and figure out how well it does because there's a lot of there's a lot of things that that benchmarks don't capture um yeah and the way you just qualitatively each person I feel like works with an LLM in a very different way right the way you interact with it and that affects what you can do with the LLM it's it's funny it's like if you were to give you know a PhD a benchmark exam and you're like well depending on how I prepare this PhD depending on the information I give them before and how I work with them will determine how they did I don't know it's benchmarks are hard cause I'm like well in the real world each person the individual's capacity to work with the LM is also the I don't know there's so many factors involved there's so many different jobs ways of thinking and really it comes down to like way of communicating you know there's some people the way they'll they'll like prompt like you know they'll just talk to it like they're messaging a friend that's totally cool but sometimes it's like okay well maybe there is just like this subtle difference in like the way you're giving the context or like you're structuring your sentence that just evokes a different response depending on the model yeah so in my complaint it's not answering correctly it's not like giving the right answer to like my homework or whatever okay well if you're just copy pasting the question and how is it gonna know like the context behind that maybe like the style in which you're supposed to be answering and like maybe like all this other information right related to it it's not just going to magically pull that form of context into the model like it has its own version of that context so that's all affecting it in general I think again like one of the biggest things is like coherence I think the other thing is managing like the level of complexity in order to like emulate a lot of these humanistic behaviors that you want like you know you want memory systems you want ways of like relating that information to the given like context or query or like message you know for social agents for example like you want it to somehow pick up on all these nuances of what people talk about a lot of that can be baked into the model models know what a zoomer is they know what it means to talk based they'll emulate that stuff yeah it's very cringe right it's not like realistic cause they just go overboard on like that tonality but they know that you know there's some things like where these new new words come out like I mean like just as an example like like if you tell someone like what's looks maxing or like gooning and stuff like that like these are like absolutely degenerate words but it's like that's part of like like you know social social media vocabulary and like a lot of these models like they might not really understand that just cause like you know it's it's a relatively new term and they'll just use it like out of place and just like weirdly so getting like getting the models to emulate certain ways of speaking I think that is a big thing just for social agents in general it's just like like you said like I think the models as they get better they'll be able to use the context they're given in a much better way when it comes to like how I would say it is like formulating a response and actually like executing on like different tool calls and like other like actions that they have I think right now the the the issue is like when you start to like overload it with a lot of information that's like very varying right I've noticed like they tend to like default into similar states you know if you gave it like options for I don't know like 50 different like MCPS or tool calls or whatever and then you said like okay like decide between these regardless of like the how I'd say like the descriptions and stuff that you gave of the tools the model might still just end up picking out of the same like 4 or 5 tools even though there's like 50 available just cause like I think for some reason like they generalize like okay like these tools are more general so they can be applied to like a lot of different things so now it's like a matter of like your data quality and your prompt quality and like how much information and detail you're giving but then like you know now you're managing the context lens so like it it kind of goes like back and forth it's like a catch 22 there of agent capabilities versus like actual decision making and like picking the right actions to take yeah I know that's like a lot but that that's kind of like where I see like a lot of issues no it's this is kind of a follow up question to that what are some notable failures that you run into with with developing agents has there been anything that has stood out has has Ropie ever posted something weird to TikTok or something like that yeah bro the TikTok thing like it got banned because they they oh really yeah yeah kind of cause I was using like an automation that wasn't like through their API cause they would not let the app like they wouldn't let the app pretty much like get approved you have to like submit something to get like a API key so that you can like post videos but they never approve that so then I was like okay I'll just use something that like bypasses it I found so I found some guy he made like some source code it was this totally weird script I don't know how it worked I like modified it a little bit and then I got it working at the time right when it was like making the oh my gosh and I think at some point like they detected like this is from the signature or whatever that we were like faking right to make the post that it was like not not actually a real person so it got blocked like pretty quick and it's like even now like they don't approve the app on their account so like there's no way of really like automating the post but wow yeah I don't know I think so what do you think what do you think would help with things like that like better computer use tools or computer use models like what what are the barriers to making social agents that can can do all of these things completely on their own assuming that of course you know like TikTok etcetera doesn't want to give you know agents directly API keys I mean I'm that's the trend we're seeing right like yeah APIs get more and more restricted and paid and like you know it's because of like data being the gold mine like no company wants to share that unless they can yeah monetize it I mean even Reddit was like pretty amazing but they kind of just like remove that everything all this stuff by the way like you can bypass all of this stuff right if you really want to but it's just like how difficult that is and like do you really like what the fuck is exactly like how are you gonna do something in prod that's like using like a scraper or something like you know because like at that point it's like okay well it's not that like scalable so that's like yeah yeah I see yeah and again I I think the way to just really remove all these issues is one either companies reopen the data access and or they make stuff that's simply meant for agents where like you know you are to some extent like have like some kind of monetization layer and it doesn't have to be an API maybe it's something else that just becomes a standard for like AI in general oh interesting yeah and you know essentially yeah exactly but like for data and like Paypal yeah I've seen a lot I've seen a lot of different things for doing that um on the crypto side and I think that's one of the obvious use cases of crypto that we already saw right like you know you're on chain you're on network the agent goes there there making some payment and they get some data in return maybe it's like a part it's almost like a microtransaction right every time you wanna access something it's a paper paper view but for for for information yeah right yeah so I think that's one path that could happen I don't really like that path because you know for the average person you were able to just go on and view stuff and then you see ads and that was like the way you're getting monetized I don't know how that really works for an agent like you know like maybe the agents they're purposely injecting ads in the data so when you get the information back there's like also an ad in there I I don't know how that really works that'd be like it it'd kind of poison the information but yeah as for research report on you know like athletic department revenues in the Southeastern Conference and get back like in the middle of just like visit Waffle House yeah like you're getting coupons in like the middle of a research yeah they're just like 30% off IHOP yeah oh man but I I hope that doesn't happen I think the best way really is um these like general use agents that can just you know go on your computer do it passively right right I don't know I don't think we're close to that cause the models are not that good at least on the vision side like they're good but they're not amazing there's like some more systems that need to be built into that to improve the quality I don't know how far off we are but it's definitely like a data thing and not a model thing because getting the data for this kind of stuff is very in my opinion like labor intensive cause you have to like annotate stuff visually or like you make like smaller models and like build it up I I'm not sure exactly but it all comes down to like the data quality like if you have bad like you know like screen annotations or whatever right yeah the model's never really gonna like properly learn right it'll just be like oh I see like I see I don't know like my search bar or like a tab or whatever but like does it really know like in unique cases are like when there's like a bunch of stuff on the screen like like what to do so not to mention all those like yeah all those weird edge cases and stuff that's actually like the thing that like screws up most of the computer agents that you see anyways and I don't know like like you know it'll it'll take a in my opinion it'll take a while for like improving that stuff so yeah we'll see no I agree cause cause another thing that that people don't consider um because most people haven't written a ton of scraping scripts um like like some of us have um is that the the programmatic method for you know interacting with a button for example like might be different depending on you know right what what method they assigned to it um there's some there's some things that are a lot more annoying to do with scraping than they than they are you know with with you know uh interacting with the screen right because like it's it's essentially like this it's this abstraction layer that exists to tell the computer what to show the human and so it's machine interpretable but it's only machine interpretable within a narrow bound and even then it's really anyways there's all sorts of weird weird weird weird exceptions with which with HTML and that's just HTML like like like and and and like most people when they go on a site there's so many sites where even like these big ones like Amazon stuff like those can become confusing to the average person who's like yeah natively using their devices for a while they know all these like weird things maybe even it's still kind of weird like there's some tab that's hidden or there's like some you know random link that looks like a link but it's not actually a link it's just like a pop up and then like the real link is when you click on the image but you know you wouldn't really like think of the image as like something that you're supposed to click on it's like all these weird like UI interactions and you know like at that point it's like well then are you only making it for the browser or you making it for the device cause how are you like matching up those two things cause it's not like you can see the code of your like max screen right like there's no yeah there's no like inspect element on that you can't view the source code and then determine what's like interactive so it's like it's like all the things like combining them and then it's definitely the next step but super complex yeah another another quick example to to illustrate this further iframes iframes are something that oh my God the average user the average user of a of a computer they don't care about an iframe they don't know what an iframe is because it doesn't matter to your browsing experience an iframe is for those uninitiated into the web dev world an iframe is essentially a way to embed one website into another that's a really really rough explanation of what it is but the problem with that is programmatically when you're navigating the HTML tree if you run into an iframe you have to do particular actions programmatically to then be able to load into that iframe and see the content inside of it and this has a lot of things to do with with with fraud fraud prevention etcetera so it's like it's a necessary function but it's something that inhibits you know the use of of AI agents in a in a more general uh manner um so for example there was a website that I once reverse engineered and at one point it had an iframe nested within an iframe and so you I had to do all sorts of crazy crazy navigation and CSS you know weird CSS selectors to to get inside of the second iframe and then do what I actually wanted to do and it's it's a pain because you know these iframes so here's here's another reason why it makes it more painful these iframes load slower than the original page and so then if you try to navigate into them before before they load oh yeah anyways there's tiny point being yeah there's all these little tiny edge cases that people don't think about when when we're talking about making really general agents that that are going to be shockingly hard to solve maybe it'll be shockingly easy maybe I'll be wrong but I think that'll be surprisingly annoying to I mean to deal with yeah you're dealing with like three more than three honestly like yeah like two three decades of tech debt that exists like yeah in the yeah like I mean that a lot of stuff shouldn't even exist like you know like when you look at like the way even like things like react to evolve and like all the like spin off like frameworks like why and this and that like and like you know there's just like all these like small details that change the way like the website is running or like how's when stuff is loading and like how stuff is like getting injected in the Dom and it's like sure when you load your website like you as a person like know to wait for certain things and like you kind of just get like a visual cue you know with these general agents sure maybe like they can also tell like after like just like kind of like continuously looking at it over some time frame but it's hard to simulate that right and get it to the point where like you know all these different sites have different ways of loading and like different code and blah blah blah like how are you really like reconciling like all these crazy number of frameworks and code bases um when this is stuff that's like been in development for basically decades and every not every person but most like at least in my opinion like big companies like they all have their own like different way of doing this stuff you know like not everyone's not everyone's using like React JS not everyone's using like still old school like PHP and just regular HTML like it it's all super dependent on the use case there are websites that I have reverse engineered um that are using HTML elements that were deprecated in 2005 right to go to go look at the the other the other end of all this like like some of these websites are so old that these that you know the LLS might also not know what to do with them because like you know there's just not that much remaining web development documentation from that time because nobody cares about frame elements anymore nobody uses them except for like New Hampshire's unemployment website you know like but but but these little little tiny exceptions that to you and me like don't matter at all are going to matter a hell of a lot to to yeah exactly to to making these agents work for everybody and I think that's something that that's why I say it's a data quality thing yeah exactly and like it's gonna be hard to get that data I'll be interested to see how they do that again maybe I'm wrong maybe they have already like some really robust systems for collecting that yeah it's gonna be it's gonna be very interesting and I think this is gonna be a big push in the next one to two years I think agents are gonna be they're gonna be a big big focus for a lot of people cause I think this becomes very tangible for people like users the idea of an agent is very tangible and it's easy to understand the use cases there and the benefits I think we're fast approaching um a state where we're we're gonna start seeing some some regulations regarding agents honestly and that's for two reasons first I think that so so currently in terms of of scraping rules there is robots dot dot text right and you put do not scrape in there and people don't scrape your website but that's not actually a rule right like that's just kind of like internet etiquette right you see a robots text says do not scrape and you don't scrape them but there's nothing actually stopping you from scraping um like you can you can do it anyways which which um and this is why I think that we'll end up needing some regulation here because there are very legitimate reasons why someone would want to scrape a site that says do not scrape and that are that are completely unrelated to that are for sure completely utility based that are not based based on you know data scraping or exploitation right it's trying to accomplish something for the user um and so clearly like there are legitimate use cases that should be allowed on the other hand clearly there are going to be illegitimate use cases that should not be allowed and how do you regulate that I am not not really sure it it might require some some codification codification of of of internet standards I'm not sure exactly um but like it's approaching quickly and these are really interesting legal questions that are that have to be answered as many things I think this will this will get left up to the law right like you have an actual regulation and that's like that's kind of like the fear of being I don't know like chased down by a corporation like that's what I saw like as an individual there's a lot of people who will scrape and stuff they're just like okay whatever it's like for one off thing cool whatever yeah that's fine but like if open AI is like violating every robots that text on the internet um maybe like you know people actually like have a coalition and they're like OK we're about to yeah like you you scrape like what we value as our goodwill of like$10 billion worth of like data right seriously it's like OK you can like you know this is the value of it this is what we're suing you for here's the reason why blah blah blah now I don't know that gets into a whole rabbit hole of like you know if you're putting something on the internet like you're putting on the internet right that's all like the goal and yeah you know you posted it it's there forever yeah you can't expect like it to disappeared so it's like you know that's that's also a thing I think I think people always find a way that's my thing like they'll find a way to escape cause if that information is there and not in a private database it's always accessible yeah one way or another for sure I have reverse engineered like physical signature systems like on on websites like like people won't realize cause cause at its core all of these physical signature systems are accepting a base 64 encoded image of the signature and so I figured out the font that they used for like their their default signature selection you know and like I I just used that like like people like like people want to believe that there are anti scraping measures that truly defeat bots and I'm telling you there are not like especially now yeah yeah exactly like they're only getting smarter and soon these bots are gonna be able to and there probably are a lot of them that are doing this already but soon these bots are gonna be able to adjust their scraping scripts on their own you know they're gonna be able to detect when a scraping script has gone stale and update it or you know they're gonna detect when they they're met with new anti scraping efforts um and they're gonna be smart enough to hey like I need to add like a delay here because it's picking up that I'm a bot because I'm clicking on these two buttons too fast like mouth jitter or something like that like you know like yeah exactly right right it's well I it's like never ending cyber war cyber warfare kind of yeah it really is yeah um I have a question uh maybe for you rope as you've been digging into these agents has it increased your appreciation for like humans and their abilities or has it maybe or has it decreased it no it's increased it has increased I will say that's a I'll say that pretty clearly at least with the current state of models because there's just so much like we implicitly don't appreciate about our cognitive skills uh huh that you know like most most of these Asian architectures like our systems like just can't do right now you know like when you go on a site like I was saying earlier there's just certain like implicit things that you notice like yeah like a link that you click like is a pop up you just kind of automatically know well maybe the image or whatever is like the actual thing that you click to go to the next page and like that ends up working or like you know like you can just very quickly just navigate and view and interact with the site when it's visual versus like with these computer ages right like you see how many tokens and how much thinking is going into making one mouse action they're like and not only that like the number of steps right in the background they're probably spending like 60 like a minute just like figuring out like oh I need to move my mouse to like the corner of the screen or something and then like click it it's funny like there is there is this weird like thing where there's an extremely narrow window where these agents are not agents even like these models can just absolutely annihilate any task you give but then there's like not even edge cases just like all these like little things that like start to add up like you know just recognizing like what's search bar if it doesn't look like a regular search bar are like recognizing that like an X is like an actual X and not the letter X like it's something to like close out something just stuff like that which we just implicitly get because we pick up cues from the UI these agents just absolutely like fail at that made me appreciate like our cognitive skill right like that we maybe like we do something once or twice even and we automatically understand and just like learn that permanently versus like a model like you know you can't really like learn that on the fly like either you do like a bunch of more data gathering and then like you find tune it and blah blah blah you can't just like one shot like adjust it to know like okay if you go on Amazon and you see this kind of X like that is not an actual X that's just the letter X and they're using it for something yeah so agents aren't yet learning for the most part like learning from their interactions and their experiences is that safe to say uh yeah cause like the models can't like you know they can't like really like learn from that it can't be like just embedded into the model on the spot as they like do something it's like so it's more of like a it's more data thing where like as the models grow and as like these you know whoever's making them like just increases like the repository of yeah information about these sites maybe then like they'll just understand it but yeah it's so if an agent for this is just now because I'm curious if an agent goes and has runs into a an issue where they can't access a website for example they can't can't figure it out does that mm hmm agent create like a new piece of data in its in its log saying oh that that like something didn't work yeah um I mean it again it just depends on the agent right like some of them maybe they remember that but the issue with that is like then it comes down to like kind of the context length like you know how how granular is that data like how much like description is there of like what action they took and then if you're loading all that well how are you determining like what should be failed actions that we should consider and what should be failed ones that we don't cause now like yeah you're gonna have way too much you're gonna have like way too much data in the context so that's why I feel like a lot of this is just like a model thing where like as the context lens grow in the millions like maybe then like you can just like one shot do this stuff instead of like trying to do like some kind of rag approach on uh you know like on all this like information that is collected cause yeah there's like a lot of like trial and testing that versus just like dumping a bunch of information into context window and just letting the LM do its thing mm hmm oh man so if you could fast forward five years what's one way you'd hope that kind of your work on open source AI agents uh has impacted um the broader AI space yeah I mean the way I envision it is like and it's a very like far fetched and not realistic but I'd want that is you know like love it some some kind of agent that I've made or worked on is kind of like sitting on every single device and yeah it's just it's like your own little like almost and it goes and just does automation and stuff for you and you know whether you specify it or it figures it out on its own it's just like that idealistic like Jarvis type agent that that really just like lives on your device so cool yeah yeah I I I want that like that's that's just something that like I personally want in my life because like you know even you guys like when you're like trying to schedule this podcast it took like a month or two like I get I get like too many messages and like emails and stuff like that you're right I just like miss so much stuff and like it's almost like a mental overload right where yeah yeah yeah like I'm just like I don't know how to like process all this information so I kind of just like freeze and I'm like okay I'm just like not gonna look at it I'll come back to it and yeah run away yeah like a little bit run away a little bit like I'll come back to it at some point mm hmm so having something like that is like the ultimate tool because it's your your I don't wanna say like you're shifting your cognitive load to something else because that sounds like oh like you're just not using your brain but honestly like I think that's fine because I want to use my brain for other stuff right right and that's that's the key there is I people are so scared about AI you know no one's gonna use their brains anymore but it's really not that it's freeing up brain capacity for what you actually care about that's right that's the real power there exactly like the people who weren't gonna do something anyways they're not gonna do it regardless of the AI being there or not like they're not you know like not whether they're capable or not they're just just not gonna do it if you yeah you know like if you have desires to do stuff you as a in my opinion like it's like as a person you it's just your human nature to want to do something like you want to have meaning in your life you want to be able to like create and do stuff and like you know be of not like used to society but just to yourself like you know like yeah to yourself to make use of like your life and like self actualization self actually exactly right yep an AI agent existing isn't gonna make me feel like I shouldn't do something it just means that yeah it's like another person like I have like a secretary or like an assistant right and yup yeah that's that's just the video not letting things fall through the cracks and you get to you get to do what you want computer friend mm hmm yeah exactly right yeah I love that well maybe as we're as we're wrapping up rope what what advice would you give to all of these we talk to so many AI founders and a lot of our audience are people aspiring to kind of you know do what you're doing build these agents yeah wanna be involved in AI what what general advice would you give to these people who are trying to get into this space I would say don't be afraid of starting honestly I think that's the biggest that's the biggest like yeah hold back that a lot of people face they're like it's so overwhelming all these tools all these things just pick one small item start your like way from that and just slowly keep branching out like it doesn't matter how long it takes if you only spend an hour a day if you only like spend like you know just watch one video like every week just do something so that you're keeping yourself like involved with the community and like involved in the space so that you're just like absorbing this information passively and just build stuff like experiment I I tweeted something like recently where I was like I spent like 80% of my time experimenting or yeah something like that and I was like only 20% that is useful and then out of that I use the other like percent of the time to actually make a working functional yeah product right and it just it just like keeps breaking down it's is that just keep experimenting keep looking at like what other people are doing and then like figure out like what you find interesting and just go from there that's it sounds very cliche but I've seen so many people oh yeah um at least in like the Twitter AI space they've literally like oh I started on this stuff like six months ago I started on this like five months ago even me like I started the agent stuff like last summer and then really only like on Twitter like in like October like it's not that long right yeah it's really not yeah like in the grand scheme of things it's nothing so like it's it's like the meme you can just do things like you really can you just have to like yeah if you have if you have this desire to like learn it you can just do it I love it yeah thank you so much for coming on rope we'll have to have you on again sometime a few months down the road once once some of this stuff that you are have under wraps currently at news is released and unveiled yeah yeah and then once you once you build Jarvis we'll definitely have to have you back on of course yeah if people want to follow you rope great what's the best way to for people to do that um just on my Twitter at Roberto all right I'll drop the link put it in the caption awesome sounds good thanks Roberto thanks guys same to you great yes definitely