When high quality effort is applied to a tool, such as AES or the linux kernel, we intuit that it "hardens" the tool. That is, it makes the tool more correct, more resilient, less assailable, etc.
Similarly, when effort is applied to an open problem, such as the Riemann hypothesis or P v NP, without progress, it "hardens" the problem: it makes the problem feel more daunting to whoever takes a stab at it next.
Andrew Wiles, whose interview also hit the homepage today (https://news.ycombinator.com/item?id=49075264), couldn't just tackle Fermat's Last Theorem head on, he had to wait until a different, modern problem reduced to it, because FLT had gathered this mystique of unassailability through its 300 years of existence.
A thing I worry about is that as AI transmutes tokens into effort, it'll split the world into two: some problems will yield, making human effort entirely unnecessary, and others will harden to the point where human effort will feel increasingly less worthwhile, because "even AI couldn't solve it". I don't like this. AI is spiky, so I suspect it'll continue having major blind spots, and yet its mere presence will probably have a chilling effect on what would have otherwise been useful human effort.
AI is nowhere near the intelligence of a very educated person that has innate talent for problem solving. It does solve the problem of applying human intelligence on problems that truly need it. AI is also a great tool to see if there's something simple that we've missed or just haven't even attempted due to wrong assumptions.
This is a problem that will solve itself, people will continue to work on the problems that AI fails at, likely by telling AI the approaches they want AI to take.
i think id almost worry more that ai can solve problems in latent space that it cant translate back to tokens because decoding ruins it, and that we wont be able to come up with concepts that we can map to properly decode those solutions in a way people understand
... what is understanding of mathematics anyway? if some AI result helps a mathematician to solve more problems I would say then that it gave them some understanding, but just as there are proofs that span hundreds of pages it's likely that soon proofs will be long Lean programs and studying them will be part of mathematics, just as studying Go played by AI.
This blog post talks in depth about what you're talking about. It may interest you. It even talks about the future where math proofs are just Lean programs, and why that won't necessarily be a good thing.
They’re likely referring to his post / thoughts on the bun rust rewrite and specifically those he shared about the lead developer of the bun project and the bun project at large.
I would not characterize what he shared as vitriolic, but from the comments when the post was submitted one would get the impression that is not a majority opinion here.
The comments here indicate a surprisingly, in my view, mixed reception, for such an unambiguously good talk?
The main thrust is simple:
- Technology is not good or bad, it is an amplifier of human values and our collective ambitions.
- Those of us who are terrified of where technology is going are actually terrified that humans are, on the net, bad, and technology will amplify that, bringing about dystopia.
- Believing this this is self-fulfilling. By embracing such beliefs you surrender your agency, which is what actually brings about dystopia. If you, instead, have a positive vision of humanity, then you are one of the people that drives society forwards rather than backwards.
We are currently at a moment in history where this message bears repeating.
A lot of things feel unstable, and as a result we have an opportunity to redefine the world in good ways and bad ones. Those who choose to redefine it in ways that are intractably bad will often do so from the point of view of a system observer, not a system participant. This is cope, because in reality everyone, whether they choose to accept it or not, is a system participant.
> The comments here indicate a surprisingly, in my view, mixed reception, for such an unambiguously good talk?
It wasn't an unambiguously good talk. Many of the people attacking various portions of the talk are making good points. I actually agree that blackpilling is a self-fulfilling belief and indulging in it weakens your own agency, and it's better to have a positive vision of humanity and to wield your own agency towards that end. But the thing about positive visions of humanity is that different people have different, mutually-incompatible positive visions; and I think a lot of people don't necessarily agree with Andrew Kelly's specific one.
I don't really think the main thrust is correct, though, if that's what it really is. Because humans don't have to be bad for tech to bring about a dystopia even under your first premise. They just have to have enough instincts that are even potentially adaptive in times of scarcity that become maladaptive in a high-tech environment for tech to bring about a dystopia. And I don't think that dystopia will be a self-fulfilling prophecy, even if your second premise is correct, because if humans have a sufficient number of instincts to interact with technology to bring about a dystopia, it's very likely to happen whether we believe it or not.
Finally, even the majority can be optimistic about humanity or even everyone, and it could still lead to a tech dystopia because of the additive effects of good intentions sometimes adding to bad (see fossil fuel use for example).
I don't think we have many opportunities to redefine the world. We're already in a state where people just trying to get by is contributing to the destruction.
> - Those of us who are terrified of where technology is going are actually terrified that humans are, on the net, bad, and technology will amplify that, bringing about dystopia.
I think this is true in that people will do what they need to in order to get ahead. If you play a game by the rules against someone who is cheating, then you will lose. The only way to stay competitive and have a chance to win is to also cheat. When people see others who act in an anti-social way (PE destroying companies, enshittification of software, etc.) go unpunished and reap massive rewards...then they too have to be anti-social just to stay on the same level or have a chance of succeeding. It's the descent into a low-trust society and a general race to the bottom.
The hope for me was that humans have learned from history, from all the past mistakes of failed civilizations and collapse of authoritarian regimes to build a modern world that is equitable. But there will always be narcissistic psychopaths who are able to say the right words and sweet talk those they need to in order to gain power and repeat history.
You're surprised they didn't eat the tokens to churn on lots of open problems instead of asking others to pay for those tokens? They're in the token business. If they're eating the tokens, it's in support of a marketing effort, not in support of innovation across the frontier of all the other academic disciplines. The collective frontier is way too big for them to just "solve it" without asking society to at least help them break even on such an enormous public good.
It's also much better to distribute the challenge of identifying problems amenable to which prompts
Could they do it? Sure, but to what end? It would make more people hate them and feel even more "take our interesting work." Pitching it as a useful tool just makes more sense on all levels
I know. I'm just thinking about how to make AI create SVGs better... in theory, a sufficiently smart AI could "generate an image in its head", think about it, and then output the SVG paths to produce said image. Intuitively that would be somewhat closer to how human artists convert artistic visions into a sequence of arm movements while holding a brush (obviously, humans don't hold a fully formed, photorealistic image in the head while drawing, but rather vague concepts, but still).
I suspect this article got upvoted partly for the headline & content, and partly because of who said it. Lots of people already have a good feel for the AI/journalism niche Simon Willison occupies, and can use that background knowledge to read into his words more productively.
I think it lands differently for different kinds of prose (e.g. informational, persuasive, etc.)
Persuasive prose, in particular, which is probably the majority of the things that get posted here, is less persuasive to the extent to which it includes obvious "AI tells". Even in the cases where AI is more articulate, (1) the emotional weight of the text feels manipulative when it is clear that the emotions were, at best, "vetted" by a human as opposed to having been produced by one, and (2) I think we have a built-in "bullshit-proximity" sensor, and have recently been trained to expect AI to be more willing to engage in bullshit than humans due to their obsequious disposition. (For the record, I think humans are also full of bullshit, so this one's more of a toss-up.)
Low-tech users don't give a damn if something has the guts of an "app" or not, they care about having a thing on the home screen they can click.
Businesses have the incentive to give their users that low friction experience (at the point of need) using already familiar rails (i.e. "install app from app store").
The makers of both iOS and Android treat the ability to "bookmark" a web URL onto your home screen as a power user feature that requires navigating through complex, technical-sounding menus. Does it have to be like that? Of course not. They just have a business interest in pushing users away from the open web and towards their walled gardens.
--
Mind you, I'm not saying, "advertising doesn't play a role in this". A clump of well aligned motivations is obviously going to be more powerful than a single isolated motivation. But let's not forget that apps built for non-technical users, which—I cannot stress this enough—IS MOST USERS, benefit greatly from lowest common denominator solutions where they never feel like they have to learn anything to get going.
> The makers of both iOS and Android treat the ability to "bookmark" a web URL onto your home screen as a power user feature that requires navigating through complex, technical-sounding menus.
Not the case on many Android browsers, you can present the user with a button to do this for them by listening for the `beforeinstallprompt` event. There are some requirements to meet for that, but it's a pretty user-friendly way to push your PWA to the home screen: https://developer.mozilla.org/en-US/docs/Web/Progressive_web...
Yes, this is the answer. It’s easy to forget if you live in a tech bubble, but there are probably billions of smartphone users who don’t even know how to type a url into a browser and navigate to it.
I’ve worked on projects where we ran this experiment, and the success rate of “install this app and click on it” is several times higher than “navigate to this webpage every time you want to use our tool”.
One small correction though, android has made it easier to add a “progressive web app” to the home screen now. You can prompt the user with a dialog asking if they want to install it. I think there was at least some period where Google was really encouraging PWAs. iOS still sucks. I’ve had very poor success rates in getting users to install our PWA using the iOS workflow (and our tool is something they need for their jobs, so they are highly motivated to install it).
We only come to understand idea quality via evolutionary fitness tests, which means we have to MAKE things in order to find out if they're any good. Everything you make is gonna do hill-climbing in a [potentially unstable] environment in search of an ecological niche. This is true of species a la Darwin, but it's also true of products & ideas.
Anything we make gets to participate in this evolutionary process, even if it sounded dumb at the outset. In fact, doing dumb things is how we get "unstuck" from outdated ways of thinking.
Similarly, when effort is applied to an open problem, such as the Riemann hypothesis or P v NP, without progress, it "hardens" the problem: it makes the problem feel more daunting to whoever takes a stab at it next.
Andrew Wiles, whose interview also hit the homepage today (https://news.ycombinator.com/item?id=49075264), couldn't just tackle Fermat's Last Theorem head on, he had to wait until a different, modern problem reduced to it, because FLT had gathered this mystique of unassailability through its 300 years of existence.
A thing I worry about is that as AI transmutes tokens into effort, it'll split the world into two: some problems will yield, making human effort entirely unnecessary, and others will harden to the point where human effort will feel increasingly less worthwhile, because "even AI couldn't solve it". I don't like this. AI is spiky, so I suspect it'll continue having major blind spots, and yet its mere presence will probably have a chilling effect on what would have otherwise been useful human effort.
reply