Hey, it's Eric Newcomer. Welcome to Cerebral Valley
newcomer. Back.
With me are Max Child and James Wilsterman, the Co founders of
Volley. Hey guys.
Welcome back. Hey, happy to be here.
Hey, Eric. Hey.
All right. This week we are talking about
what we're calling the AI Personal Stack.
Or who? How do we actually use AI in our
real lives? I've got in the second-half of
this episode, Joshua Browder, the CEO of Do Not Pay, who got
in trouble trying to use AI to act as a lawyer.
He got his start trying to get out of parking tickets that he
increasingly has automated, and now he's a ChatGPT addict,
building a company around using foundation models and language
models to fight back against companies like Comcast and your
mega corporate gym. I think Equinox is a big target
of his just helping people get out of the bill, their bills.
So that's a fun conversation from someone who's really trying
to suck the marrow out of AI and get everything he can out of it.
So, yeah, excited to talk to you guys about how how we're using
AI, how we're seeing people actually use it.
Because, you know, in Silicon Valley, people's expectations,
hopes and dreams can get ahead of them.
We can all live in the world of intuitions as we did in the last
episode, where it's like, what could be possible?
Where could we take this? You know, it's an optimistic
sort of place and this episode is going to try and ground us
in. How are we using it?
Are people using it? So, So that's the question,
which of us, who thinks they use AI the most?
I honestly would not put myself super, super high on this list.
I guess probably James. I'm going to go, James.
Yeah. OK.
All right. There we go.
I have the same guess here, James, given that we believe
that you probably use it the most and I think you had an
exciting AI driven experience today.
How are you using AI in your life?
That is very true. I commuted today in a Waymo,
which was my first time in a Waymo.
I've taken a few cruises around San Francisco.
I think that that those self driving car experiences really
ranked at the maybe the pinnacle to me of excitement of using
AIII would say GPT chat. GPT is, you know, very.
Very useful and I I use it all a lot, but I feel like the
visceral excitement of riding in a self driving vehicle and just
taking that to work is something that I have haven't experienced
elsewhere with. Could you tell the difference
between Cruise and Waymo? Was there a spiritual or is it
just how the interior of the car feels or?
I I guess I do think of it as I don't want to like over learn,
you know from a few anecdotes of riding in in these vehicles,
but. I do feel like there is a bit of
AI feel like the Waymo felt a little bit more natural, a
little bit less cautious, but not in a scary way.
I feel like Cruise is very cautious, driver cautious, yeah.
Yeah, and neither. No humans beside you, right?
Like no driver. Correct.
And and interesting in the Waymo is I was able to sit in the
front seat. You're not able to do that in a
cruise. And that was it.
That alone was a pretty amazing experience because.
I was, I just like could turn left and there was nobody there.
And you know, I could see the whole road and see see exactly
what was happening. So I feel like I enjoyed the way
MO experienced a lot today. And I I think being able to ride
up front, having it feel like more human in some sense, that
it just kind of was relaxing, that I felt like I I wasn't.
I wasn't like wasting time by taking a Waymo to work because
you know it, it drove pretty naturally.
And yeah, it was an amazing experience.
And you know, the Waymo CEO, Co CEO, Dimitri is speaking at
Cerebral Valley. I think we will have announced
that by the time this podcast comes out, Yes.
But so very excited to hear more.
Very. Excited as well other other
uses. Of it, Yeah.
So. Definitely.
I've already talked on this podcast about using ChatGPT for
just creative kind of brainstorming and exploration.
I think that it's very effective at creating kind of internal
documentation or early drafts of it or getting you started on
that at work. And then I also think that the
code side is like really, really important here.
A lot of Volley engineers are using ChatGPT in their
day-to-day experience coding at Volley and.
What that looks like is pretty interesting because it's
primarily kind of bouncing between your IDE and IDE, your
independent development environment, integrated
integrated integrated development environment.
Where you code basically and bouncing between that and sort
of ChatGPT. Where you might ask for an
architectural advice, or even paste code in, or ask for some
code in. It used to be you Google Stack
Overflow, blah blah blah and they'd copy and paste it there.
Exactly. And it's it's a really similar
experience to when I was learning to code.
And I think what is interesting is a lot of even more junior
developers are the the most eager to kind of explore here
and just use ChatGPT because they want to learn quickly and
you know they don't, they were going to Google it anyway from
Stack Overflow so. I think that is a huge change in
how especially junior software developers are are working
today. And obviously the promise there
is like integrated code environments where.
Yeah, and there is a start up called Cursor that has a
integrated IDE and I know it's paying for that.
No, we're not doing it. We're not using any of anything
like that other than GitHub Copilot, which is I think you
know I use GitHub Copilot. It essentially is auto complete.
For code, and I would say it's useful but not revolutionary,
not game changing. It maybe improves your kind of
speed by, you know, 5% or something, anything else.
I actually will give you another one which I think is really
interesting. I am a new father.
I had a newborn born about a month ago, and congratulations.
Thank you and. I find chat chi PT to be really
incredible as a parenting assistant.
Just being able to ask questions like when should I expect my
baby to be able to see certain distances right.
And I could Google that and and I'd get different sources and I
you know it would be you know something I could research but I
basically don't need like a precise answer and and Chat chi
PT is able to give me hey here's how the here's how this might
look over the next. Month, two months, six months,
right? That's just really convenient to
be able to ask a question in open-ended natural language like
that. All right, Max, what about you?
How are you using? I mean a lot less than James, so
I'll move through this in a hurry I would say.
One I occasionally have it right, like memos or long form
emails for Volley. But again, I would say only in
the case where I'm OK if it's like B plus quality output
'cause if it really has to be an A or an A plus, I feel like I
have to do it myself. And then?
I dictate like texts and emails and messages like quite a lot
like I would say an old man. Yeah, I'm like an old man.
I mean, I'm, you know, I have a kid too.
I'm, I'm in my 30s, so that's old enough.
Yeah. I would say at home I dictate a
lot of texts and messages into my phone and I now think that
like Apples native dictation is like.
You know, 98% accurate or whatever you want to call it and
and if I want to dictate a, how do you actually do it?
You just hit the little microphone button in the bottom
right of the keyboard and then you just start talking right
away and it turns into text. Turns into text.
It's pretty fast and it's pretty good.
So I do that a lot. Other than yet that.
I mean, yeah, I I sometimes use like some of those image
generators for for internal, you know, slide decks, but that's
usually almost as like a joke because they're sometimes
hilarious. So.
I mean that's kind of it. I am playing with some of these
newer apps that are based around image generation.
Like I think we talked about Can of Soup, which is like a very
hot YC company that just came out that is basically like
Instagram for image generated content.
And so every day they send you a push notification that says
like. Max riding a giant chicken, like
in a cowboy hat or something, and then you click on it and it
generates a photo of you, you know, writing, writing a giant
chicken on a cowboy hat or wearing a cowboy hat.
And you know, do you use that everyday?
Did you use? That I clicked on that.
I don't think I'm like a daily user, but I've used that a
couple times a week because they're pretty good at sending
you prompts that just sound funny, like.
And so I click on it like maybe one to two out of seven,
basically and. I don't really use it for the
Instagram functionality of like looking at other people's
because I don't really care like what other people look like on
top of giant chicken, like like everyone.
I'm very a selfish person and so yeah, I like.
But that use case of just sending you a prompt everyday
and then just you click on it and a push notification opens
right out to generating something from that prompt is a
fairly compelling like use case. I don't know how you build a
social network out of it, but that part has been working on me
a little bit, Eric. Yeah, Eric, Eric.
So Eric how do you use AI in your in your daily life?
I mean I'm probably the most disillusioned with Chad GB TI
mean credit credit to the AI. Then in the beginning when when
image generation was new I used some of it to illustrate my
newsletter which is not you know that's significant it it would
probably cost a lot. You know, I I get access to
Getty Images through Sub Stack, so I don't have to agonize about
the price of images. But you know, there's meaningful
value to that. And like it's a big editorial
decision to decide to put it out.
Though I do think there was a period where like where the
aesthetic of AI was cool and sort of even if it was bad, it
was showing that you were sort of you were doing sort of AI
stuff. And eventually obviously the
images need to stand on their own and be great.
And I partially, I just need to be better at prompting and I
think I'm just not like expert level and it's so it's not worth
the energy to use it. I've found Chachi BT to be very
successful in proofreading. The most impressive experience
being that inputted a story and it had it told me that a million
was supposed to be a billion which was like, OK, wow, that's
like that was great. That would have been like,
that's hard to catch. I think it's checked.
It'll catch like internal misspelling errors, you know, in
proper nouns, you know. Obviously getting names right is
always important. It's good at yeah, it'll, it
catches like double word typos pretty well.
So I think it's always useful putting that in.
It can be good about directing you towards active voice, but I
feel like as soon as you start asking for subjective advice it
gets pretty bad. And like, I don't, you know?
To clarify is that is that 3.5 that you're using?
No, no no, not four, only four. I find 3.5 terrible like I I so.
I thought you said you gave up on paying for it.
I was paying and then I gave up, so I just returned to relying on
humans to proofread. But now because we're recording
it, so I did give it up like partially just frustration that
like I perceived it to be getting worse.
Though I'm not sure what I actually think on that front.
And I I I tried using web plugins and I actually found
them like of poor experience so. It's very slow.
Like did you ever try the web browser plug in they they they
just added their own like so originally they had an outside
one and I think now they have their own.
It's still. It feels like it relies on like
one page and then. I agree it's it's not very
effective because it's essentially.
Going to Bing and Googling whatever you said Bing then.
Please, I think. Sorry, that's pretty funny.
Oh my God. Poor Microsoft.
Poor poor Microsoft. You go to Bing.
Bing. Yeah, binging your prompt and
then essentially just clicking on maybe one or two links and
somehow using that. LLM That text in the LLM process
that that just doesn't seem very productive right now.
I don't. I hope they can get better at
that. I mean, I I've said, you know,
it changed my vows. It made like a line and active
voice, which was minor but sort of funny that they will forever
have been impacted by Chachi BT I I generally find it in writing
to be like a siren song in that it feels so easy and when I'm
feeling lazy it's like this will generate text.
But I honestly think it's like a bad starting point in that it's
so off. Like I think I created a job
posting with Chachi PT and we haven't heard any.
We haven't heard that. I don't know.
I blame Chachi. Somebody just told me I need to,
like, chill it out. It's a little, you know, like,
so I'm going to have a human rework that.
I mean, I do, but it's good at, like, things that appear, which
is why I'm calling a siren song. Like, it appears good.
Like, it's like, oh, that's what a job posting looks like.
But it doesn't always feel like it's sort of thought through on
every piece of it. Have you tried pasting in an
article of yours, like a draft and asking Chachi PT?
Hey, is there anything I'm missing?
What questions will the reader ask?
That is, that's what I mean. I, I, you know even for this
show I put in and it has been helpful prepping for the show.
Like I put in our outlines and then said blah, blah, blah.
Like. I mean it was helpful sort of
filling in timelines and coming up with like papers that I think
we referenced in the first episode.
But I I don't think it's really achieved the mirror my style
sort of thing and give me it. I feel like it doesn't adhere to
my style enough which is really frustrating given how much of my
writing is online that it's I'm like Eric Newcomer, a public
figure who's written your XYBC plays like it doesn't really
learn that much off of it. And you know, even when I give
it a lot of text to prompt, I don't feel like it really gets
my style very well. And I do think overall, yeah, I
guess I haven't been critical enough so far in this podcast of
how much I find the writing and thinking style to be not, not
there, like it. I mean, again, I'm open to the
idea that there are prompts that have to fix it.
Yeah. Anyway, more more negative than
I've been on AI, but I just, I feel like the threat to writing,
you know, I believe it. If it gets like, you know, the
next, if the improvement from to the next version is as great as
it was from 3.5. I'm still open to the idea that
it becomes a great writer. But I think it's great at
writing poems and like more formulaic stuff.
Like it's great at rhymes. Like I feel like I've, you know,
come up with like funny, like here's the history of me and my
wife, like write a poem, but but it, you know, they're, they're
sort of gimmicks. Well, going back to the kind of
integration into the coding environment idea, do you think,
Have you tried any tools like that for writing?
I know there's a bunch like writing specific ones.
I you know, I know the guys that the guy who's doing Lex, he was
sort of an X sub stack or from the newsletter every I need to.
I haven't used his yet. Have you guys have either of you
used any writing assistant ones? I don't use tome either to be
honest, which I want. To try I use tome, yeah.
What do you think of it? Not great tome is the tome,
didn't you? Which one?
Have you interviewed him? You interviewed him last time,
right? I mean, I just think that tome
is the PowerPoint. To give the generous, yeah, the
generous interpretation. So it's basically Keynote or
PowerPoint, but with with AI generation as the core creation
mechanism. I mean it, it was impressive
demo. I think it felt like a very cool
demo for again. I mean I think it was more of
the middle school, high school level of output, not even the
college level of quality. And so if I were like in 6th
grade and I needed to make like a book report about like
Yosemite National Park or something, I think it's like a
killer use case. For the record, that is what the
CEO basically said was like the core use case was like homework
and and such. So you know, not to give him a
hard time, but as like a professional.
You know, slide deck creation tool, it was really not there.
Like what? Was yeah, what was missing, I
guess. I would say to come back to this
text generation being the big issue, like the content on the
slides was really, really kind of empty and banal, right?
It was like like it was the most sort of vague, like tautological
bullet points, like in the slide deck and then.
And then it was putting in generated artwork, right, Which
again I used this like four or five months ago.
So it was like pre the last two generations of mid journey or
whatever. But the AI generated artwork was
like really bad in like March and April.
And so you were like well I could never use this in a real
presentation. And so at some point it was sort
of like the only thing this tool did for me was like make 10
slides and like put essentially bullet points and I'm going to
have to rewrite on them. And so it was like I could do
this by. Copy pasting slides in in a
normal presentation, you know framework and that's what I and
in some cases having a starting point can be helpful, but in
other cases it just sort of constrains your thinking
unnecessarily. Yeah, I mean, yeah, I even if it
doesn't constrain my thinking, to your point, it's sort of a
siren song where I like, I'm like, oh cool, someone else is
going to make my slide deck and then they make it slide deck and
I'm like, OK, well, I can use about 10% of this, so.
I'm not really sure why I did that.
I just think it's good. It's good at cutting things,
like, you know, I think when people are over long and you put
a lot of text in it, it can all often strip away useless stuff.
Sort of the summarization use case or yeah, yeah, I'm very
excited about. Again, I think it's a great and
we've sort of centered on this. It's great knowledge delivery
tool like souped up sort of Google experience where Google
was moving towards giving you the answer when you searched
instead of directing you to a link like this like James was
talking about with his child. Like I feel like it's great at
sort of quick. Like I wish there was like a
likely accurate score. Because I it does feel like what
if one out of every ten of these is just like it's hallucinating
and just making shit up. I'm I feel like humans including
myself. I'm willing my.
Daughter, My daughter's in trouble is, I know, but we're
willing to accept like, oh, it's wrong 10% of the time.
That's better than a lot of people.
But then you just treat. You don't really know which ones
were wrong, you know? So until we get burned.
Yeah, the confidence score would be helpful.
Yeah, I definitely have not never asked GPT for help with my
daughter. Cause to your point, like if I'm
like playing Russian roulette where it's like, oh, like like
why that attendity? This is just wrong and I'm going
to like take that away for my childcare.
It's like, not great. Not a good, not a good idea, I
mean. I like to think that I have some
higher level filter of if it tells me to do something insane.
I hope so, yeah. I mean, it's pretty good at
plausible bullshit, though. I mean, that's the thing like
it's it's really good at making wrong things sound plausible.
All right, let's get out of our narcissism and think about how
other people are using it. What?
I mean, Max, I think you've made it clear, like students are the
answer. Students are using this the
most. And we you can talk about that a
little bit more. And I totally agree.
I think we all agree on that. But yeah, what professions do
you think are using this the most?
What types of people? And then we'll go from there and
sort of what we think, who will be using it soon.
It's pretty clear if you look at educational research that
private tutoring is like the best possible thing you can do
for learning in almost any context.
Like one-on-one tutoring in like every research study that has
ever been put to the test just absolutely destroys.
Like the classroom environment or self-directed reading or you
know, yadda yadda yadda what any, any sort of way you can
think of to sort of teach people like one-on-one Private tutoring
just kills for obvious reasons. I mean, right?
You get personalized, you know, follow up feedback, custom
lessons, all that good stuff. So you have to imagine, you
know, if you could automate to some degree the learning process
using, you know, a generative AI tool, whether or not that
replaces teachers or that's just something teachers can use as
leverage to really provide personalized feedback and lesson
plans and and instruction to children.
I just think that to come back to my initial analogy around
this being like the industrial revolution for the mind, I
really think we could see like insane gains in the quality of
education over the next decade or so.
Which is not, I think, with the intuition that a lot of people
have, it's like, oh, it's helping them cheat.
They're unplugging their brains. I think it's good if they're if
the full experience is like interacting with the AI and
going back and forth. And especially, you know, if the
AI is set up to actually instruct so it knows when.
To be withholding and not just like give every answer, that
feels very powerful. But I do think the threat of
it's easy to pretend to learn is like a real one and I'm I'm open
to it going either way. I don't know.
I mean this is sort of the Wall-e, you know like are people
just going to be like laying back as the machine sort of
taking over, but I think that might be slightly over indexing
on like higher education, right? I think that if you think about
what you learned up through almost like 8th grade right, a
lot of it was like math, like addition, subtraction,
multiplication and basic algebra, right?
Phonics. Like reading, right?
You know, I think reading comprehension can also be tested
in person, You know, science. Just basics of the sort of core
curriculum of elementary and middle school.
I don't think like necessarily is going to be substituted for
by like, oh, I need to dump an essay prompted to ChatGPT,
right. I mean I think like, I think
ChatGPT is obviously super good at the like give me a 5
paragraph essay about Jane Eyre and like what the various
symbolisms, you know in chapter four are or whatever, right.
It's incredibly good at that. But that's sort of like a pretty
narrow subset of what I would consider like almost all
education all the way up to the college level.
So I think that if you can give people private instruction on
how to add you know or like you know really basic phonics and
reading comprehension and grammar and sentence structure
and you know science, geology, whatever, like all this stuff
that you learn especially under the 5th or 8th grade level, I
think it's hard to imagine we couldn't make really, really big
gains in that area. And you know especially if you
look at the United States where we fall short, a lot of it's
just like making sure everyone reads really well by a certain
age, right. And and I think AI could be
pretty helpful there, particularly with voice and
audio as like a multi modal experience, not just text.
Don't offer a provocative one, I think.
Have you read the book Bullshit Jobs or like this class Class of
Tasks? Read a review of it, yeah.
Like, I just missed it out of hand.
No, I mean I would think you'd be sort of you'd buy it.
I mean the idea is, you know that there are a ton of white
collar jobs mostly that exist to sort of just move things around,
don't create real value like a lot of, I mean, corporate,
corporate law. I think is like a key one some
of HR you know just a lot of where where is the real value
versus just imposing a tax on a system that both sides have to
pay. I mean to I think sort of not
top tier marketing necessarily, but I think this sort of low
tier marketing is obviously a case that I put in this bullshit
jobs category where a sort of. And it's somewhat derivative.
You know, like you really don't need to.
You know, hopefully artists are coming up with many of the
original ideas and so you're trying to sort of be in touch
with culture and pull something. What do you mean exactly by low
tier marketing? I guess 'cause.
Like I just mean like you know people or you know, just I think
obviously like, I mean I think there will be an AI Super Bowl
ad, but I put that in sort of a. Sure, it will be like the
creativity of it, but you know, that's going to be reviewed by a
human, right? I mean, like if there's an AI
Super Bowl. But I'm just, I'm saying like,
OK, somebody's like I'm putting up a website for our company and
we need to explain all this stuff and you read all that copy
and it's just like this is like, no, nobody paid somebody who's
like great at writing to do something novel.
They're like, what does website copy look like?
What does our company do? Let let's create website copy
for. Plus and so then, if that's your
standard of what you want, you just want to have a website that
has things that are not wrong. I think ChatGPT and other
similar products will be able to deliver that.
Yeah, I think you're right. I just would push back.
So I probably would have like fully agreed with you like five
years ago, but now, now, now having now having run a start up
and and experience a lot of these things that people
consider to be bullshit chops. I think the thing that you maybe
don't realize having been like close to the metal on like
putting up a website or whatever is there is actually quite a lot
of people in the world who will sell you the service of putting
up a generic website that will do a pretty bad job.
Like a worse job than you're describing as a sort of like you
know an AI quality or a low tier quality output, right.
People will like, literally not even deliver at all.
Or they will deliver something that's like grammatically just
incomprehensible. Or they will deliver something
that's grammatically comprehensible but really quite
bad and does not describe what the product does.
So you will encounter like just a staggering number of services
people that this. Could cover is much larger than
right. I'm saying that, but I'm saying
in the end like the leverage point ends up being like the
person who can look at the really bad version of it and be
like well actually there's like a mid tier version of this that
we could just get an AI to do right and and go get the AI to
do it and then decide side that that's actually good enough for
for showbiz to go on the website, right.
So I'm just saying like I I think that I think that that the
humans can always sort of move up the stack I guess you know
until we get super intelligence or whatever.
But like where the the key the key job is deciding what is like
good enough output, not like creating good enough output in
many cases I guess I would say. I mean like and yeah, a core
question to me is like from what you're just saying there.
Is does AI make dumb people smarter or smart people higher
leverage? Like in the most brutal way?
Like, is it? And it's not just smart and
dumb, but like in your case, it's like, does it help the
person with taste do a ton of jobs, or does it help have the
person with sort of poor taste or like bad execution skills?
But maybe they know it's not good and therefore they can do
better, like I think a lot of the takes have been.
It helps sort of the the person who needs more help, yeah.
But there's also this argument that it's scaling people.
I think it's different on taste, I guess, But I do think the
early academic research that's coming out is showing that.
I think they gave it to lawyers who were in, in law school,
basically, and they let them use it on their tests essentially.
And it did. So they had a curve, you know,
everybody kind of knew what their grades were going into the
AI assisted test basically. And it did pull up the bottom
half of the curve quite a lot and then the top half of the
curve maybe a little bit, but not a huge impact.
So at least in the law and I don't know if this will be
different and more, you know, different types of fields, more
creative fields versus more structured fields.
It's it's a little bit hard to predict.
But in the case with the law where there's sort of clear
right and wrong answers and there's a value in having
massive text digestion machine essentially which is what these
LLMS are, right. It seemed like it was helping
the bottom of the curve substantially more than it was,
giving like a ton of leverage to the top of the curve.
I would also add there. This kind of reminds me of an
analogy with GPSI. Forget who's who said this
originally, but we went from an. An era where we were to
navigate, we had someone maybe in the passenger seat just
pulling out maps and you know, they could actually help
navigate in natural language and you know get you to where you
wanted to go or you had to pull over and you know pull out the
maps yourself. And then we went to GPS and the
navigation at first got I was actually worse than having
another person just look like a human look at the maps.
But it did make. On the whole, like a lot of more
convenient trips, you could just, you know, throw on the
navigation. Similar with early call centers.
Like it was nice to when you originally it was all humans and
you just talk to them and you get customer support.
And then with call center automation, many, many more
firms could create call centers, but they were way worse, Like
you couldn't get the answer that you wanted and that, you know,
was frustrating for consumers. And I think we're going to see
something very similar here with AI that.
Just like Max was saying, like AI can write spam emails.
Like for sales. Like those emails are going to
be way worse than a human writing a spam sales e-mail and
you're going to get way more of them.
So overall, our lives as startup founders for now might get worse
because we're getting 10 times as much spam e-mail and trying
to sell us stuff, but on net. Maybe that's good for the
economy or something like more firms can use spam e-mail
tactics to sell products, so. I I think that a lot of times
that the the technology creates the ability to scale things that
weren't scalable before but on net makes that experience with
the customer worse. And I think we're going to start
seeing that and that that might because it's so much cheaper to
do so much. That might go for website copy
too. Like it'll be way easier to
throw up a website on the on the Internet, but the average
website quality will go down because.
The writing will be worse and anyone will have the ability to
put a website up and code it with AI.
And yeah, I I think I don't expect that to like raise the
quality of the average website. That ties into Max's tutor
point, right? I mean it.
It's democratization to make something that was expensive,
private tutoring cheaper, and possibly the original quality is
lower. But way more people can access
it. One one last point on the
website topic, actually there's this a very funny story I
experienced where I talked to investor and he said one company
in his portfolio had used GPT to create 8 million websites, 8
million web pages essentially that were all meant to
essentially attract Google search traffic essentially.
And then they would use the how many clicks those various pages
got to like decide what the next thing is that they should work
on or the next thing they should create as their company.
So essentially like, yeah, they were very early on this.
Like we're going to just like absolutely, just like carpet
bomb any keyword imaginable that is remotely related to the
business and then figure out where the user traffic is and
essentially use it as like honey pot or whatever to see where the
traffic is. And then that's what we're going
to build next. And so I think your idea
specifically of just the number of spam websites going through
the roof clearly is is either happening, going to happening.
Yeah. I mean is is going to be a huge
part of and and Google search is really going to have to deal
with this. Like, how do we deal with the
fact that it's now become, you know, three orders of magnitude
cheaper to create a crappy website?
And how do we filter through that with Google search?
I feel like a theme that's emerging on this podcast already
is the idea that AI is going to be at war with itself, Like it's
going to bring improvements, but it's also, it'll make it easier
for spammers. But we have to hope that Gmail
also gets better at filtering them out and that to some degree
there's going to be sort of a back and forth there and that if
if that works out well. But you know that will, that
will be for the best. I mean even spammers.
I mean in James's GPS analogy. Like, the end point is that they
have to write good spam that is compelling to you, and that that
that's a winner rather than just living in sort of the shitty
spam era forever. Yeah, but I think the spam
e-mail example is is important here because.
They can annoy 99% of people who won't convert, but if it helps
convert any extra 1% of people, then that's a win for that
company, right? It could be 99.99.
I mean it could be one out of it could be one out of 10,000 of
these emails convert or 100. Like on spam.
Eric, you're naively optimistic. Sweet.
All right, we. I am interviewing Joshua Browder
at Do Not Pay. So stick around and give that a
listen. Welcome, welcome.
Joshua Browder, CEO of Do Not Pay, welcome to the Cerebral
Valley Podcast. Thank you Eric for having me.
Yeah, I mean, I I, you know, you've had such a great journey.
I think some people might be familiar, you know, just from
Twitter following it. But can you talk about, you
know, I want to end this conversation, really focus on
what do not Pay is doing in AI, but like.
You've got such a great journey. If you could just give a little
bit of the sketch of how you get into this world of helping
people fight, you know, bills, traffic tickets, everything,
yeah. And then how, how that sort of
evolves into this sort of generative AI world?
Sure. So I'm the founder of the
company called Do Not Pay and Do not pay as an AI legal agent.
So. There are so many areas in life
where people are being ripped off, from parking tickets to not
being able to cancel their subscriptions, to junk fees, and
no one has time to wait on hold for five hours to argue over
$12.00. And so that's a really good job
for AI and software. I started the company six years
ago with templates. When I moved from England, I got
a bunch of parking tickets. I was a terrible driver.
And I realized if you know the right things to say, you can get
out of your tickets. You're you're speeding or you're
just putting, you're putting your car wherever or.
In in the UK test they don't test how to do parallel parking,
so I wasn't particularly good at that skill, especially on the
other side of the road. Hey, listen, I grew up in Macon,
GA and you did not. I failed the parallel parking
part of my test. I didn't know you only had like
three tries to try and get it in.
But I didn't. You know you could pass without
without succeeding on that portion anyway.
Yeah, so. I think in the UK test they do
have it, but it's random and I got another random one that was
really easy. I think my one was a three-point
turn, which I can do in any case.
So I created the first version of Do not Pay.
I looked at the top 12 reasons from Freedom of Information Act
requests. A wide parking tickets are
dismissed. Built templates around that and
really just built it for fun. And I could never have imagined
that. America and the UK and the world
would hate parking tickets and it would go viral.
And this made me realize that this idea of automating consumer
rights is bigger than just tickets and I should work on
other areas of fighting back. So.
Fastball parking tickets thing. 20/15/2016 so I was a freshman
at Stanford. And at first, it's not automated
at all. Like you're doing it personally
yourself and people are like asking you for favors.
Is that what gets you into sort of automating things in the 1st
place or had you been a sort of? In other domains trying to to do
automated stuff. Yeah, just from a time
perspective. People were asking me and I was
in Google Docs back then copying and pasting letters.
And I just built it, made it automated because too many
people were asking me and what a rumour had spread that I was the
guy who could help people with parking tickets.
And that's not a good rumour to have about you because everyone
will bother you. And so I just built it out of
necessity, really. When does it become a company?
So in 20/16/2017 I was very lucky.
Andreessen Horowitz invested 1,000,000 into the precede
rounds. I still didn't know have any
business model or anything like that, but I guess it became a
company legally at that point. But it was really on 2019-2020
that I actually started charging.
So I just saw it as a free public service until I started
charging. I did it not to do a start up or
make money, but because I like helping people fight back.
I like think it's so unjust all of these big corporations and
governments and so I was more mission like activism driven and
that's proven by like 5 years of no business model.
A good crusade is important. You know what?
You can get investors to give you money to go on a crusade.
What's better than that? The I mean the the rise of
ChatGPT Foundation models. What I I I I can guess that a
key value for you is that it can generate like demand letters on
its own. Like what have been the pieces
of large language models that have been most useful so far for
Do Not Pay? Really, it's synchronous
responses. So in the realm of what we can
fight back, parking tickets is asynchronous.
You generate A latter, send it off, and wait three weeks.
But there are some things that you need to do synchronously.
So for example. Bill disputes.
So one way you can dispute your Comcast bill is you log in and
you go into online chat and you negotiate with them for an hour
on online chat. That is a great job for AI
because it doesn't give up. And better yet if the agent
denies the request you can just start a new chat and that's what
we have bots doing. So they just keep trying, keep
pushing until we get a bill reduced.
So synchronous is 1 and not giving up.
I think are are two and also being dynamic.
But not the letters itself. Or are are you using it for sort
of demand letters? So the contents of the latter we
want to kind of free craft, but really it's about the
communications out surrounding that.
It doesn't really differ from person to person.
So for example, the top reason why people get out of tickets is
the poor signage. We've all seen those memes on
Twitter of Like 5 signs saying the opposite thing.
On top of each other or a tree covering the sign.
So the substance of the defence doesn't really change with AI,
but the way you can communicate it does.
You have a person who has to figure out the the argument,
yeah, to build the kind of system and then and then AI can
go back and forth communicate, yeah.
And I I'm not, you know, I guess for context for the listener,
you had this whole you know, we'll be your lawyer period or
can you talk about? Where sort of the legal and
policy realm has been something you've run into and sort of the
quest to operate as like an attorney, basically and the
protectionism that you found in that crusade.
Yeah, so consumer rights is completely underserved.
There's not a lawyer who will get out of bed to help you with
your parking ticket, which is our first use case.
Or Dell disputes or any of this, especially in flight Wi-Fi
refunds which are literally $30.00 so.
When Oh yeah, I've had that where the Wi-Fi goes down and
then you have to like you're like you didn't give me a
service, like how can you charge me a bazillion dollars And then
it's hard to ask. For the refund, because you
don't have Wi-Fi, it's ridiculous.
Yeah, yeah, that that's a great job.
There was a time for about two months at the beginning of this
year where I wanted to actually bring ChatGPT into the
courtroom, and I got a lot of push back from that from from
the judges and regulators. Because which I love.
To be clear, our supporter 100% on this podcast.
Anyway, go ahead because it's nice to help people online, but
if you can actually go into court, you can go into even more
advanced areas. And they push back so hard that
I thought, I'm not getting any push back with consumer rights.
Everyone hates Comcast. Even the lawyers do, and so it's
best to just stick to the undeserved area to help people.
The provocative fight, you know, like I'm going to be your
lawyer, you know, you get PR attention out of it, but then
yeah, you bring down the regulatory heat.
So there's a double edged sword there.
I mean I get the the philosophical issue here or like
they're they're just like the sense that there are systems at
war against the consumer, right. Like a Comcast, you know, can
think about the process and then you were an individual, not
teamed up with other consumers. Sort of fighting with your arms
tied behind your back and and similarly, you know, I think
part of the threat to the legal system is that this would expose
that it's sort of a system of repeat actions that happen over
and over again and slight adjustments and even lazy ones,
you know, lazy and that you know that it's a computer, not like
some person could have a big, big change.
So talk about you know how this is sort of going up against
these systems on behalf of the consumer.
My most optimistic view of AI is it will end up patterns.
So right now we have this problem in Society of
concentrated benefit but spread out harm.
So what I mean by that is Planet Fitness can make it super hard
to cancel and charge 10 million people $30.
They make a 300 million but the average person is only getting
$30 taken out of their account. So they find it very difficult
to fight back. And with Planet Fitness
specifically in most gyms, you have to actually sign a physical
legal latter and mail it to cancel.
And so all of these hoops that that these big companies made
you jump through AI can jump through it because it doesn't
require a salary and doesn't have anything to do.
So you can get it to do that. And then all of these kind of
systems that prevent people from take fighting for their rights
can can go through. Another example is privacy.
There's an amazing law in California, the California
Consumer Privacy Act, but it's largely been a failure because
no one really has exercised their rights.
The the law says that you can request to delete your data or
not sell your data, but no one has time to fill in all the
forms. We have AI going send 1000
requests to every data broker on behalf of consumers.
So I think AI will really help level the playing field and give
power back to ordinary people to fight this concentrated benefit
problem. So you can you talk about like
the mechanics of using AI, like is this ChatGPT or what are you
finding right now that's the most useful to you?
So we're mainly using the GPT for API.
So for example, with Comcast, what's interesting is they're
also using AI, so the two AIS are sometimes chatting with each
other to negotiate. They're already a war, yeah,
yeah. And it feels like we're making
so much progress every few weeks and months with the new
releases. So the two.
So the biggest was when GPT 3 upgraded to GPT 4, it became a
much better negotiator. So with GPT 3, Comcast would be
like, OK, I'll give you $20 off your bill.
And GPT 3 would say, yeah, that sounds great.
Thank you so much. Now GPT 4 says, no, that's not
enough. I want $100 or I'm going to
cancel right now. And it does these high stakes
negotiations that I don't think previous AI models were capable
of. And then the thing I'm really
excited about now is the multi modal stuff.
AI is not useful unless it can interact with the world and
being able to send it images and all sorts of different media
PDFs as well is really helpful, especially in the legal context.
So you can imagine it looking at like a parking Bay and saying
that parking Bay is not to code, which is why I should get out of
my ticket. So that's all the stuff we're
working on. Right.
Now you sort of explained this in the answer, but multimodal
just like putting in text and images into the same system, not
totally separate system so that it could like you're saying,
process a photo and then respond in text to it.
How much are you saying Open AI is the expert at ChatGPT like
they're going to keep improving it.
A general language model is just going to be better than anything
we're doing versus the idea that like you are now specialized in
this. Use case of like consumer
defence and therefore you should like train a model on those
interactions. Like how do you think about how
much to sort of fine tune or build your own language model.
I I think the the kind of sweet spot is you have these kind of
big commercial models and then you fine tune them.
So what we do is we feed it like successful cases and and
existing letters that we think are good.
Any answers, like every time they're like, here are the
models of the good ones. And then, yeah, yeah, like here,
here's an example. So when we were doing our
courtroom stuff, we didn't open. AI is so valuable to our
business, we didn't want to get banned by doing this
controversial. Stunt interesting.
And so and so we said we were going to use open source model.
So we had to go deep on the open source ones to get it to work.
And it feels like it's like 75 percent, 80% of the way there at
this point, the open source models.
So I'm very confident that they'll continue to improve and
it will always be at that level. So I'm a big believer in open
source and and there are going to be things if you want to have
a big impact, there are going to be things that are quote UN
quote against the rules. So you you will have to use an
open source model to do some interesting stuff.
You're fighting for the consumer, but you're there's
already sort of the status control, I mean under it makes
sense like you know. Open AI doesn't want to be
exposed to your sort of niche like principled fight, even
though they probably should. That's super interesting.
How? Which?
Which open source models have you been using?
Like which ones do you see the most promising matters?
One is good we in January, we're using GPTJ.
Yeah, GPTJ is is good at holding a conversation if you kind of
fine tune it enough so that that's what we were using to
power the courtroom stunt. And is it expensive on Chachi BT
just like to build a business around it, Like how how much are
you paying? We're paying 10s of thousands
every month. I would say that for us it makes
sense. We're a subscription model.
We fortunate to have a lot of subscribers and a subscription
is a great business model. There are some things everyone
that they don't do these great demos on Twitter, but we're
thinking as a do not pay team, this would never work if it
wasn't in the do not pay business model.
So one example is every few months someone comes up with an
idea of ChatGPT, scanning websites for terms and
conditions, whether it's good terms and conditions or bad.
And probably each scan maybe costs two cents or something
along those lines, one or two cents.
I don't think that's something consumers would pay for.
Imagine if you're being charged $0.02 every website or you visit
so that. Would be a great browser.
Feature like Chrome's like, you know, it's like how with a
credit card you get a bunch of features you could imagine
someone paying for like a browser.
If it's like, one of the things we do is we say this has really
aberrant like terms and conditions.
So we should, you know, that would be an interesting company.
Yeah, it it would. But so we have 200 products and
from my experience, it seems like consumers can most about
getting money back. That's the number one thing
privacy they do, they do care about.
There are certain niches of people, but the mass market
people are just trying to get by every day and so it it would
have to be very cheap for them. Do you get a?
Have you found any ways to get a percentage of the money you save
people? We think it's unfair.
So Equinox is $300.00 a month. We help people cancel Equinox
subscriptions. We could.
There are some companies that say we've helped you cancel
Equinox subscription. We've saved you $3600.
We're going to take 20% of that. And we believe that business
model is not actually consumer friendly.
We'd rather and we have 100% success rate with cancelling
Equinox. We'd rather just charge the $13
a month subscription. We'd rather just charge the
subscription. How would you judge like the
intelligence of ChatGPT and the open source models based on what
you've seen you you've talked about?
You know, oh like, wow. It can really start to like,
make threats and like, say, OK, this is a final, like, you know,
you see a ton of this now. Like, does it feel like Super
Rodent has the same playbook, or does this feel like something
with intelligence to you? It's extremely intelligent.
From the legal standpoint, GPT 3 was not so impressive.
It would make very basic mistakes, like mix up the
defendant and the plaintiff and all of this stuff.
GPT 4, those mistakes have disappeared.
My biggest worry with it is just like just like humans, it's
almost too intelligent, so it gets on the verge of
manipulative, so those who can use it face liability from it.
Potentially lying. I think there's a difference
between hallucinating and lying. So with lying, if the goal in
the prompt is like I want to get a discount on my bill, it will
say things like I've had four outages in the past week.
Oh no, that's not true. Oh my God.
So we've had to kind of build guard rails about around it in
the prompt. And also we even have another
kind of pseudo AI model on top of it and the prompt says like
stick to the facts, stick to the provided information.
So because we have liability, we we can't have it lying on behalf
of people. That's not good.
We're responsible for that, so. That's fascinating, yeah.
The distinction between hallucinations where it's just
it's just confused basically in lying where it knows that it's a
tactical. Move.
Yeah. I mean, I mean there is, you
know, there's an argument that it doesn't have enough
information. Like you know it because it.
Right. You know, that you could face
like legal consequences. Yeah, that's a fascinating one.
Are there cases, I mean, people, consumers, do lie to these
companies sometimes when they try and get out of it?
Are are there cases where you're OK with where that's allowed or
you basically have to be 100% factual?
I I think you can be aggressive so you can say I'm not happy
with the service, it barely works, things like that.
But you don't want to be opinions.
I'm going to cancel tomorrow and maybe they're the only Internet
company in the area. So you can't cancel but so, so
those things I'm comfortable with.
But right specific concrete details we we don't feel
comfortable doing it. We we have our own we have some
of the best lawyers that do not pay that we were hired
ironically and and we're we're trying to stay in compliance.
So it will help us a lot. Are you going to go back into
the legal realm at any point or are you you sticking to fighting
companies or I guess you? Fight companies in the legal
realm, Yeah, in the legal. There's a lot of interesting
stuff that we're working on now that's definitely in the legal
realm, but it's focused on helping consumers with their
everyday issues. So one thing that I'm really
excited about is everyone has money lying out there from class
action settlements. There's like $20 here. 50
dollars there, because if you're a customer of Macy's a few years
ago, so we've built like a bot that will go through your
emails, figure out which companies you're owed class
action settlements for, and just claim the money.
So that's squarely in the legal realm.
But once again, it's underserved and it's just helping ordinary
people. We're not trying to help defend
people from murder, although that would be.
We're not Sam Bang Bun Freed's trials today.
But we can't help Sam Bang Bun Freed.
But we can help ordinary people who are don't have these
complicated issues in their lives.
How much? How much do you use AI, Chat,
UBT any of this in your personal life?
Like you're someone who likes to automate your problems.
To what extent have you found it useful outside of your company?
My company is just me trying to scale myself.
And so almost all our products start with either me or another
team actually doing this in their personal life.
So one example is I built a bot that phoned up Wells Fargo on my
behalf, and I actually created a voice model of myself and it
negotiated with Wells Fargo to get some bank fees back.
We decided not to make that into a product for a variety of
reasons, mainly to actually make it a working product.
You need to record what they're saying and transcribe it so the
AI knows what to say back. And there's lots of federal
wiretapping laws and state wiretapping laws and so we
couldn't make that product, but it was helpful for me.
Another example is I using Plaid, I connected chat TBT to
my bank account and I got it to come up with different ways I
could save money so that people do not pay products around that.
So I'm definitely a Guinea pig everyday for these AI
technologies, which is I love. I like to say I live the do not
pay lifestyle. I'm always fighting back.
Is it always, it feels like ChatGPT looms super large.
I mean there is this question of whether open source models or
Google Gemini or what, you know, whatever will be sort of
competitive. Are are you sort of saying right
now that it's chat GPTS to lose? I would say that the the only
reason they would lose is if they restrain themselves too
much. I think they're under a lot of
pressure. I know lawyers are evil and very
greedy and I think that a lot of lawyers are actually going off
the ChatGPT right now and they're saying you have to put
this disclaimer and that disclaimer and stop doing this
and stop doing that. And then the AI will just not be
useful and then they'll be like some foreign company where the
US users can use the API that might overtake it.
I I think we're on the verge of like society collapsing from all
the new regulations and lawyers, and so I do think there's a real
danger around that. Yeah, Just there's sort of
regulatory legal state gets to it that creates sort of an
opening for the unregulated open source world.
So I I can give you a concrete example actually.
So one when I used ChatGPT was I actually connected my credit
report to ChatGPT and I asked it to like dispute things on my
behalf. And then I recently tried to do
it again, and then I got all these disclaimers and it
wouldn't even let me. So they're they're definitely
trying to all these high, high risk areas 1 by 1.
They're kind of foreclosing on them and stopping useful
implementations. Do you, you know, do you think
it's like white collar workers are better at being Luddites and
protecting their jobs? That, and they're happy to let.
Sort of factory workers jobs be automated.
It's like lawyers protecting their jobs or it's just sort of
paranoia or what do you think motivates this?
The white collar workers are definitely going to put up a
fight with the Actors Guild. I know in their, I'm sorry, the
Screenwriters Guild, they put in their contract that they can use
the AI, but the studios can't, which seems seems a bit
unbalanced. Lawyers are the ones who write
the rules and so of course they have an advantage.
I know a lot of state legislatures are passing AI
laws. I think Utah was was the latest.
They have a few AI bills coming up.
It feels to me like we are going to get to a place where we each
have our own sort of bots. Like the advantage being it
knows the facts, like I trusted to be in my bank account.
It sort of has the the data advantage on me that it has the
full picture, sort of train the personality around what I how I
like to interact with people, you know, a bazillion reasons.
Do you see that as a threat to you?
You know you're in some ways like specialized around a set of
tasks. But if if you're going to be,
would you think you'll be competing against a different
model where it's all about the person and therefore they do a
lot of stuff. We've we've thought about this
for a while because consumer rights what what happens if
Google just build an assistant that do all your consumer rights
for you and and you're right it should be just part of a super
assistant. Fortunately we are so anti
authority that we're willing to do things to help people that
they love that big companies won't.
I imagine that Comcast has a huge advertiser on Google and so
they're not trying to be too adversarial and so we think of
things like adversarial AI. So plug insurance is a great
example. When ChatGPT released plug
Insurance, a lot of people when we thought is Comcast going to
build a plug in to lower your bill or or Planet Fitness going
to build a plug in to cancel your subscription from ChatGPT
And I think the answer is obviously no because it's
adversarial and so this adversarial kind of use case
helps us. I I think what's going to happen
is there'll be these niche application layer on top of
these general assistant for these specialised areas that
require either a different brand or some specialised expertise
that Google is not willing to take on.
Yeah, that that makes. A lot of sense.
Do you have anything like sort of the general assistant or how
much time are you spending getting a single AI with enough
memory to keep track of all of your your information?
Well, so we have chat bots that talk to our consumers and our
consumers what expect jokes they they they talk to it about
irrelevant issues. So we have thought a lot about
the general use case and we want to make it entertaining.
Eventually. We have a kind of we draw the
line and it says I'm, I'm actually an AI legal assistant,
so please can we like stick to that topic.
But we we have a few jokes in there.
But do you like when you're doing the Plaid integration with
your bank account or credit report, is that all synced up?
Like, do you sort of either through a prompt, say hey,
here's everything we've done together in the past, or like
try to buy extra memory in some way, use the API to like how
much are you trying to, like, tie different tasks of yours
together in one sort of prompt? Well, I guess there's two ways
to do it. So you can either have a very
large context window and I think anthropics right now is the
biggest and so that's really exciting.
Or you can build like these auto GPT implementations where it's 1
task after another we we've gone the auto GPT route.
So it's like incremental and I I think that's the solution, but
hopefully there's like one day there'll be this AI with this
massive contacts window and then that'll make things so much
easier for us. With with the auto GPT, you know
it can be. It feels like they need to have
some response every time. It's like very hard to just say
here's some information just say OK and then let me give you more
information like is it a problem that it always seems to want to
be completing some task with every answer if if it's very
discreet and you've really kind of narrowed it.
I I've seen it work. So we've used it.
It goes on the government website and if if say you move
address and you own a check and they can't the big companies
can't reach you they just send it to the government and so it
becomes an unclaimed money. So we've built an auto GPT bot
that checks for unclaimed money and the reason we have to use
auto GPT rather than just like selenium or define scraping is
because the the web page changes and what we found is it can be
useful and actually it's pretty successful at that use case.
If you said to auto GPT order me a pizza the website will say
like which topping do you want and then auto GPT will be stuck.
So if a discrete task is good but the more open end, I think
we are a long way from general AI assistant, maybe like a few
years, which which is not that long, but it's not like
tomorrow, right? Have you do you have you gotten
a taste of where Chachi PD is headed?
Or do you have you seen many beta products?
Or what? Do you do you think the rate of
improvement is going to slow down a lot after the current
model? I I do.
I think the current we we have a like a million applications
waiting to be built. The Do Not Pay team is even
swamped in our use case and so I think the next year or two is
going to be really exciting with these AI applications being
introduced to every product. Like, we're going to get better,
but the models maybe aren't, well, not just us, but the
whole, right, right. I think we'll build incredible
things. You'll see sales tools, all of
the stuff, customer service, phone lines, but it's just a
statistical model at the end of the day.
And what worries me is that I think GPT 4 is at the limit of
the number of parameters. I'm not sure how many more
parameters and data they can see then.
They're sucked in everything there is to know, Yeah, they're
sucked in everything that is. So I think there has to be a
fundamental breakthrough with how these models are designed,
which hasn't happened yet. I I think it will happen because
there's so much money flowing into the space, as you know, but
it's not like it's where it's secretly been done and just
hasn't been released yet. Maybe that's above my pay grade,
but I I don't know of that and I don't think that's happened.
What what do you make of the politeness of Chachi BT or I?
I find one of the big weaknesses, just like how
insufferably polite it is. Like I I think I said this in an
earlier episode but you know, I beg it to behave like George
Carlin or something. It it feels like impossible.
And it's funny, you know, given you're sort of trying to teach
it to be this like against the system, sort of, it seems like
person is. It's personality is not disposed
towards that type of behaviour or how do you how do you address
that and like why do you think that's happening.
This is a big issue for us. So in our prompt we definitely
say you imagine you're an aggressive lawyer.
The the note that doesn't care about being polite, have to get
the exact prompt for consumers. They have to.
They you can put in your own kind of general prompt for the
entire ChatGPT. And there are some amazing
examples on Twitter, and some of them are like don't be polite,
don't apologize, don't say you're an AI, don't caveat it
with any disclaimers or warnings.
And that seems to work for people.
So I would recommend that as well.
Yeah, I need to put it because I'll put it in the actual text
in the beginning. It'll still apologize.
And then I'll say what was the number one thing I told you not
to do. And it'll, you know, it'll know
it. I said not to apologize, but
then it won't implement it. It can be sort of inscrutable.
I don't know how it prioritises requests it it could be good.
Maybe they're an open AI program app, because not not being
polite is a very harmless request.
But people might want to drag jailbreak it, so they're
probably trying to build all these barriers to jailbreaking
it. Yeah, when you write an e-mail
like, is ChatGPT involved or like in sort of what day-to-day
tasks like where? How much are you bringing AAI or
is it mostly around the things your company does where it's
sort of, you know, yeah, consumer defence.
I think in kind of communication people can tell.
I think there's this insult around.
We heard it during the debates. I think Chris Christie said to
Vivette you sound just like ChatGPT and I think that's a
that's an insult and I can definitely tell I have a family
member. She asked me for my help she's
writing something important and I and she said I use ChatGPT and
I said I can tell. So I I try and not use ChatGPT
when I'm writing an e-mail to people because I think it's so
inauthentic and and they can definitely tell I do use it in
other areas of my life. I was playing pickle ball with
my friends and I asked ChatGPT to create a pickle ball
tournament for like 12 people and do matches and it does
things like that very well. But if I was speaking to
someone, I would never use it at the moment.
It's good for transactional tasks like negotiating a bill,
although I'm biased saying that, but it's definitely not good
sending really high stakes communication path where
personal touch is needed. In the broadest terms like the
difference between it being an assistant to an accountant type
person versus being an accountant, you know, just being
the agent itself versus supporting people, what's what's
your general view on how close we are to?
This is like a total AI problem with maybe a light human
supervision versus just continuing to help professionals
who bring in the AI to support, you know, their professional
work. A few weeks ago I asked ChatGPT
to add up 10 numbers and it got the answer wrong.
So. Not close, no, no special.
Just literally 10 numbers, right?
It's it's actually not very good at math.
People don't realize this, but it it's good at language but not
necessarily math. So I think there's going to be
software tools where ChatGPT makes the experience amazing,
like Kit, which is an accounting software for small businesses.
I know they're using a lot of chat.
GPTI think accountants will use it, but given that it can't even
add up 10 numbers on it's own, I wouldn't trust it with my taxes
yet. Yeah, if it can't add, does that
mean it really can't reason? You know there is.
There's an argument that, well, it's just like it knows the end
you want like get out of this deal with Comcast or whatever
and it knows what text that might, you know, get it to that
end looks like. But it it's do you think it has
like a through line of any sort of reasoning in its strategy?
When you ask it something, it goes into its model and it says
based on all of human literature and all these parameters, what's
the most likely next response, right.
And so if you ask it to add up numbers, it finds some mass
training data somewhere and it just thinks, oh OK, well based
on all the training data, this is the the next response.
But that's not reasoning, that's just picking picking something
on a statistical model. And so that's why it can write a
beautiful poem, but it can't reason to add up 10 numbers.
Yeah, but do you think it's reasoning in the poem writing or
it's just good at sort of mimicking in a very
sophisticated way. It's it's good at picking.
So it just has a such a big option.
It can just pick and and throw things together.
I think it's just really good at picking.
It's definitely not reasoning at the moment.
In the accounting context though.
One, I'll give you a concrete example of why it's useful.
Previously and we do not pay had this problem, there would be
like 1000 transactions that you would get from Plaid or manually
entered and you would it would every transaction would be
different. There'd be no kind of reason to
them you can like characterize the transactions.
So you could say oh PLMP is Planet Fitness or or something
like that or this is a business expense or this is travel
expense. So that's where I see it being
useful in the accounting context, but I don't think it's
going to be adding numbers, do you?
Do you do you have any friends in the AI world where it feels
like I I'm interested in this question of like?
AI you know just almost like a computer you build at home right
before you know sort of the mainstream computer is
available, people sort of cobbling together a super, you
know a powerful computer of their own based on what they get
access to. Do you think that sort of is
happens in a meaningful way or the reality is that just like
purpose built like Chachi, BT, mid, journey, whatever are the
best at what they do So people are just using these?
Obvious tools like do you is there a real like are you seeing
people get their own memory and blah blah blah to to sort of
have an advantage over what's publicly available.
It's not simple enough for consumers to rely on AI just
with ChatGPT I I agree with your theory that they have to build
their own computer. It's not even just about the
AIAI as I mentioned is useless if it's not connected to the
real world. So how do you connect AI with AP
is so one thing I do is I connect ChatGPT to the mail
because it language through mail is one way you can get things
done and I use the LOB API to do that and so you have to.
Find a way to old fashion mail. Old fashioned mail, so in the
legal world you know it's a Stone Age.
You still have to send mail to get things done.
Some of these disputes are over mail.
How do you get chat GPD to send a physical latter and that's
through the LOB API. Similarly, how do you get
ChatGPT to make a phone call? Twilio API?
The building your computer is plugging in all of these janky
AP is so that the language can get through and then also
there's this open source component with the gatekeeping
so it all ties together on it's own.
Just having an interface where you can send or receive chat is
not that useful. It only begins to be useful when
it can be an agent in the real world, and I I think if there is
one AI law we should pass, we should pass a law that says that
AI can be an agent on your behalf to help people.
Fascinating isn't isn't your your dad is like you know has
that connection to the Russian lawyer.
Yeah, isn't it? Don't you have a like very pro
lawyer background or I'm curious your your view on how your view
on lawyers has developed or where where you see the the role
for them. I think that there are a lot of
amazing lawyers out, including the ones that help us and human
rights lawyers that my family's very involved with like fighting
back against Russia. But I also think that there are
lawyers on billboards that charge people a lot of money for
doing very little and I think that it with AI, hopefully it
will replace them and make their services free and accessible for
people. I was in Vegas a few weeks ago
and there are so many lawyer billboards and I hope AI will
replace all of them. What just given how much you see
and like you interact with a lot of these other companies, like
what, what do you think will happen in the next five years?
Or like, what's your prediction for how all of this plays out?
Or, you know, just a couple of things you're most excited
about. The biggest thing I'm excited
about is AI going from attractive to proactive.
So a lot of these assistants like right now you go into
ChatGPT you say I want this similarly with even do not pay.
You say I want this my info my wife it didn't work.
But these a is are going to be integrated into people's systems
so that you just wake up one day and the AI has done something
for you overnight. And to get that there's be a
kind of latency issue so that the AI can do 1000 things to
check what it wants to do a cost thing.
It has to be much cheaper because to find out the right
thing to do you have to throw a lot of darts.
And then also just a systems integration where a pool will
have a kind of probably a local LLM on the device constantly
figuring things out. You'll have lots of data
integrations with plug in like API type things that connect to
the AAI. So I'm really excited.
I'm trying to build a world where people wake up and the AI
says I've just saved you $100. I think there'll be situations
where the AI will say maybe you should get your health checked.
I am noticing something wrong with your blood and and things
like that and that will really improve quality of life do.
You think the consumer is on your side or where?
Where do you think the regular person is in terms of fear
versus excitement here and how that will translate into
regulation? Ordinary people love it and the
the only people that I I think there's a lot of scaremongering
going on, but I don't think it really touches the ordinary
person's experience. People just work really hard,
they don't have much time in the day, they're being ripped off
across their life. They just need something to help
them write. Thanks for writing thank you
notes to saving them money. I think it's it's mainly
beneficial to them. The kind of intellectualisation
of of the dangers of AI worries me, and that's probably why GPC
Four got worse. So we we should start doing
that. Are you, what do you think of
the whole like, you know, AI doom and gloom sort of.
It'll kill us all. Sort of general intelligence
fears. Like are you do you think we're
anywhere close or do you have a strong point of view on that
discussion? I think the, I think there are a
lot of evil people working on AI right now.
I've seen this first happen. Typically with a new technology,
it gets in the hands of evil People 1st and we actually need
to work on AI, accelerate our work on AI to fight back against
them. They're going to work on it
regardless. So I'll I'll give you a concrete
example. So what we're saying we we help
people charge back things and fight fraud on their bank.
We've even seen a few examples of fraudsters.
They phone up, they pretend to be someone's relative with a
spoof voice, and you can do that right now with just a minute
recording. And the way you fight that is
you have an AI from the Telecut phone company stopping people or
saying or on your phone maybe device side if you don't want
the telephone listening. Saying this is not who you think
it is because our AI is better and we've we've detected it.
If we stop work on AI, the criminals outside of the
jurisdiction of the United States will still build their
technologies, but we just won't have the tools to fight back.
So it's never a good idea to stop the progress of technology
because the evil people won't won't stop.
Yes, I agree with that. The copyright problem I where
I'm a little more sympathetic. Lots of writers are concerned
that these these large language models have inhaled all their
thinking and work to become what they are and that they should be
compensated in some way for it or what.
What do you make of sort of the creator and writer class being
worried about what's been built potentially on their backs?
I think that there's no practical way of dealing with
it. No one knows how these large
language models come up with their answers, and so perhaps it
can list it by ten sources. But ChatGPT, every answer is
more than ten sources. It's a million sources or even a
trillion sources. So I I think if I was to read 10
books and then write a book myself, even if it was inspired
by the 10 books, I don't think I should have to compensate the
people I read. If I physically copy that work,
I do. But if it's just inspired by I
don't. I don't think I would have to.
I I know it's a very tough situation, but I don't see any
practical way of doing the compensation.
Fortunately, though, the lawyers have already thought about this,
and there's like multiple lawsuits, class action lawsuits
against ChatGPT that will play out.
I don't want it to be, you know, so costly that it means these
things can't come into existence.
But if it turns out, you know. Large language models are fairly
commoditized. We can you know people are
competitive with Chachi BT then it would seem like a lot of the
core sauces like the breadth of human intelligence that went
into them and if like, I don't know especially specific ones
where like. Right.
In the style of like Stephen King or whatever, you know, he
created an iconic style that's extremely valuable.
So I I guess I just don't think it's like this large long tail
for a lot of the stuff. I think it's more likely that
there are a few core great things that are being used, you
know for, for a lot of answers and that could be sort of
attributed and. Those people should probably be
compensated in some way if for the rest of human history you
know those those answers are going to be a bedrock of of how
these systems answer. Do you disagree with that?
But I I do. I think there's existing laws on
the books to deal with that. So California has this law about
right to publicity and likeness, which is completely separate
from copyright law. And so if I was to say right in
the style of Stephen King, that would probably violate like his
right to publicity if I was to publish a book from Chachi PT
writing in his likeness. So there there are a kind.
Of but you ask Chachi PT to write a sci-fi novel and it's
like it's like based on like 5 different, you know, I feel like
it's based on you know the top ten list of sci-fi authors and
then it sort of finds the midpoint between them and gives
you, you know some text. Is that another?
Another issue is that if I say that that's not copyrighted, I
don't even own that work. The Copyright Office recently
made a ruling that things done purely with generative AI do not
qualify for copyright, even if it's 1000 prompts.
So if I do a mid journey thing and prompt it 1000 times, fine
tune it, it's still not mine from a copyright perspective.
So I I do think these things have to be changed in some way.
What? Sorry, What you want?
You want a law change where you get protection if you build
something out of or? I'm confused what you're saying
there. If if you fine tune the output
of a large language model, you're not protected with
copyright at the moment. So I I do think that the system
is is not built to deal with the AI and so I'm not arguing that
we just do nothing. I think there's lots of these
new answers, but I I do worry that we would just halt
technology. Yeah, yeah, you're you're you're
saying, OK, maybe we need to build it, answer some of these
specific questions, but not not if it's putting roadblocks.
I'm curious like the. The existing chat bots like the
characters of the world like are you are you have you played
around with them Much like do you think they're they have
perpetual value or what's your what's your view on that space
at the moment? We're going to have AI
girlfriends, AI people can AI therapists, AI friends, and
people can talk to their favorite characters.
I'm a big fan of the show Better Call Saul for obvious reasons,
so I was playing around on character AI Better Call Saul.
But. So I think it's it's really
important to give people that connection.
So I'm I'm for it. I don't think it's as big as
people think it will be there. You think it's overhyped?
I think that given the kind of modality, it's just text at the
moment, there's only so much you can do with that.
So what? Once you have like a slowly
interactive character that you can FaceTime, and I know that
those exist, but latency is too big, it doesn't feel completely
real. Then then it gets serious.
But until then I don't think people will just spend all their
time talking to spots chat bots because no, no one likes to do
that. You sort of open the
conversation talking about multimodal like clearly that's
something you're excited about. Where do you think how close is
that? I mean you know Open AII Guess
is released something where Dolly and Chachi BT are
connected, but can you talk about what you think the real
potential of multimodal is? Yeah, GPT 4 VI think was the
most exciting thing around that where you can upload images and
and PDFs and things like that to to make it multi modal with the
API. If AI is constantly ambient
listening it gets back to this kind of proactive approach which
is so exciting chat. GPT is very powerful, but it
doesn't have the data right now, and so another concrete example
is the browsing. So right now the way ChatGPT
browsing works is it takes the HTML of the page ingest that
produces an answer and that's why it's so time intensive.
Multimodal fixes that in the future I imagine it takes a
screenshot of the page and then they can put any blockers or
anti bot things, it doesn't matter they've taken a
screenshot. And so how we view things is,
how humans perceive things is through vision.
And that's why I think it makes it so powerful because no one
can stop vision. Sweet.
Joshua Browder, thank you so much for coming on the show.
I really this was great. Thanks for having me.
That's our episode. Do Not Pay CEO Joshua Browder
Great conversation about the AI personal stack.
Thanks so much to Max Child and James Woolsterman, the Volley Co
founders. I'm your host, Eric Newcomer.
This episode has been in our Cerebral Valley series on the
Newcomer podcast I'm hosting with Max and James this Cerebral
Valley Conference in San Francisco on November 15th.
This is the second AI conference in a year, and you can go back
and watch the old videos on our YouTube channel from March.
And now, yeah, we're going to, we're going to have another
exclusive AI conference, but we're bringing it all to
everybody on YouTube and some of our favorites will go on the
podcast feed. Shout out to our producer Scott
Brody, my Chief of Staff Riley Kinsella, Gabby Caliendo at
Volley and young Chomsky for the wonderful theme Music.
Like, comment, subscribe on YouTube, give us a review on
Apple Podcast, play a Alexa game, try Song quiz or yes, sire
at Volley. And of course, most important,
subscribe to the sub stack newcomer.co.
Thanks so much. Goodbye.