ATProto World logoATProto World

Menu

John Herrman (@jwherrman.bsky.social) profile picture
View on Bluesky

John Herrman

@jwherrman.bsky.social

7.7Kfollowers
547following
573posts

posting about posts at new york magazine. have me on your podcast!

Top posts

John Herrman avatar
John HerrmanยทAug 19

just checking in with one of the most influential philosophers alive today

Post image
198
187
2.2K
John Herrman avatar
John HerrmanยทJun 26

The real AI arms race is between one another ๐Ÿฅฐ nymag.com/intelligence...

Stories about Al deployment tend to fall into a few categories. You've got productivity stories, where workers โ€” most visibly at tech companies โ€” talk about how Al tools are making parts of their jobs easier or harder, increasing their workload or simply making them redundant and taking their jobs. You've got top-down management stories, where Al use is suggested or mandated by leaders demanding more efficiency, who are either betting that a great deal of automation is possible within their firms, or who are just worried about getting left behind. Then you've got the stories in which people are more clearly using new Al tools against one another in an escalatory way. Job hunters, now able to generate custom applications instantly, flood employers, so employers turn to Al to manage the glut. Spammers and other bad-faith actors flood social media with near-infinite material, pushing the platforms to double down on automated moderation. Rapidly generated presentations lead to rapidly scheduled meetings recorded and automatically transcribed by AI assistants for machine summarization and analysis. Dating-app users generate chats with Al only to be filtered and then responded to by someone else using AI. The starkest and most consequential such story is what's happening in education: Teachers dealing with students who generate entire essays and assignments are turning to Al-powered plagiarism detectors, or getting pitched on ed-tech software that solves cheating with surveillance - with, of course, the help of AI.
These are stories about AI, but they're also stories about broken systems. Students flocking to ChatGPT in the classroom suggests that they see school in terms of arbitrary tasks and attainment rather than education. The widespread use of Al in job hunting drives home the extent to which platforms like LinkedIn, which promises to connect job seekers with employers, have instead installed themselves between them, pushing both sides to either pay up or dishonestly game their systems. A dating app where users see opportunity in automated flirting must already be a pretty grim space. If Facebook can be so quickly and thoroughly overwhelmed by Al-generated imagery and bots, it probably wasn't much of a social network anymore โ€” a low-trust platform better at monetizing users than connecting them. Smaller-scale AI arms races like these don't take hold unless users (or workers, or students) have already been pitted against one another by systems they don't respect. In an uncomfortably large portion of modern life - especially online - that's exactly what's happened.
20
159
559

Latest posts

John Herrman avatar
John Herrmanยท1d

So many bizarre AI products are like this but it's really important to understand why: Companies like OpenAI want the world to give in to their total success before they've actually achieved it nymag.com/intelligence...

As I spent time watching ChatGPT bonk its way through various web interfaces, I also found myself thinking of self-driving cars. A browser that pretends to be a person at the input level - moving a cursor, scrolling a human GUI - felt less like a Waymo, in which an unattended steering wheel turns as a result of actions taken by systems closer to the road, than a regular car with a humanoid robot sitting in the driver's seat. Again, it's pretty interesting to watch! But it also makes you wonder: Why are we doing this that way? Aren't there better ways for machines to talk to each other?
The answer, Miller suggests, comes down to the "incentives of their makers more than the intrinsic value of the technology." (The Browser Company was recently acquired by software-maker Atlassian.) For OpenAI, building systems that can execute complex commands on behalf of users is the whole ball game - it's the path to wide-ranging automation and/or AGI, depending on which definition of the term the company is going with that day. An AI model that can provide useful information in a chat window, or handle tasks clearly outlined by the user, is a useful product, but the prospect of an Al model that can productively and proactively interact with the world around it in ways comparable to a human - or, more specifically, an employee โ€” is where trillion-dollar valuations come from. To get where it wants to go, though, OpenAl has a number of challenges. Some are widely discussed and frustratingly hard to pin down, revolving around benchmarks, varying definitions of model capability, and predictions about scaling. Others are more banal: To answer questions more usefully, for example, chatbots tend to do better if they have more data about users; likewise, to execute tasks on their behalf, they need to operate in an environment where users are logged in to the various services they use to work and live their lives. They need a breathtaking amount of access and permission, in other words. ChatGPT isolated in a chat window doesn't have that, and it takes a long time to draw out of users, if they're willing to offer it at all. ChatGPT as a default browser - authenticated in dozens of different sites, payment methods at the ready, or perhaps even logged into a work environment - does. (Such agents also create, as many in the AI space have pointed out, a potential security nightmare.)
2
6
17
John Herrman avatar
John Herrmanยท3d

one way to understand media coverage of the second trump administration is that, this time around, it's really only tracking with the bottom right chart

8
89
312
John Herrman avatar
John Herrmanยท4d

Interesting how basically every Amazon project โ€”ย true of most tech neo-conglomerates, to be fairย โ€” comes back around to surveillance nymag.com/intelligence...

In various nearby industries, Amazon is regarded as a leader in employee surveillance and "algorithmic management" and is likewise held up as an example of its possible effects on workers (more productive, less content, and less likely to organize). In a brief rendering demonstrating how its delivery glasses will work, Amazon shows how worker surveillance will soon extend outside of the vehicle, right up to the doorstep. Which brings us to the second bit of news, from TechCrunch: Amazon's surveillance camera maker Ring announced a partnership on Thursday with Flock, a maker of AI-powered surveillance cameras that share footage with law enforcement. Now agencies that use Flock can request that Ring doorbell users share footage to help with "evidence collection and investigative work." Ring cameras are an underrated Amazon success story: With an appealing pitch โ€” see who or what is on your doorstep, even if you're not home โ€” the company sold millions of units and constructed a massive surveillance network with numerous benefits for the company itself. They're a way to counteract package theft and to make it easier for customers to receive deliveries.
They also provide an additional way for the company and its customers to surveil its workers, meaning that Amazon's glasses aren't just extending its "end-to-end" delivery apparatus โ€” they're closing the loop in its employee-monitoring ecosystem. Ring cameras were, for obvious reasons, always interesting to law-enforcement agencies, with which Amazon has had a generally cooperative but heavily scrutinized and limited relationship, at least until recently. Now, by partnering with companies like Axon and Flock, which operate nationwide fleets of license-plate scanners and work with local, state, and federal law-enforcement agencies, including ICE, the company is making its surveillance network widely and comprehensively available to the state.
1
12
23
John Herrman avatar
John HerrmanยทOct 23

๐Ÿ˜…

0
0
3
John Herrman avatar
John HerrmanยทOct 23

Deep research tools *did* get better after I wrote this but I wonder if people might be revisiting their rapturous reviews now that the novelty has worn off. A few months in, DR outputs remain technically impressive but they're also often... unreadable dogshit? Not useful? nymag.com/intelligence...

1
2
13
John Herrman avatar
John HerrmanยทOct 16

if you want a picture of the future, imagine your own face staring back at you from the chumbox, forever nymag.com/intelligence...

Virtual try-ons are actually showing up all over the place in retail โ€” you'll find them on both luxury-brand apps and Shein โ€” and they tend to suffer from the same issues, given that they're essentially mashing images together: The clothes don't fit quite right; the lighting is always strange; generations can take a little while; body types are approximated but rarely nailed. But they might be good enough to help rule something out or consider something new, and they're certainly good enough to be compelling and occasionally fun to use. The temptation to merge virtual try-ons with ads โ€” to follow users around the internet not just with evidence that you know a lot about them but with actual photos of themselves using new products โ€” is surely massive but is currently constrained by two things. First, generating such images or videos is computationally and literally expensive. And second, putting users' faces in, say, a branded Instagram Reel would probably feel to many like a violation. (For now, Google's explainer about its virtual try-on feature marks a boundary far more conservative than that, emphasizing that the feature won't currently show up on sponsored products.) ๅ‡ธ
One of those constraints will probably go away as AI models become more efficient. The other is a matter of expectations and norms around privacy, which, if the last two decades are any guide, have a tendency to either erode or get bulldozed. Given the hunger for return on AI investment and the close relationship between the advertising industry and AI - Google and Meta, for example, are advertising and AI giants, and it's hard to imagine the latter holding out too long on a new technology that might improve ad performance โ€” it's probably a matter of time before someone breaks the seal and starts using customers to market to themselves at scale. Small companies are already trying something weird and uncomfortable. The bigger ones probably aren't far behind.
0
2
5
John Herrman avatar
John HerrmanยทOct 14

people don't like being told what they can't post, but they REALLY don't like being told what they can't generate nymag.com/intelligence...

a collage of negative reviews of Sora focused on its "guardrails"
2
4
21
John Herrman avatar
John HerrmanยทOct 10

trying to think through what's unusual about so many AI startups, and about the gaps between their pitches, the real world, and what they would actually need to succeed nymag.com/intelligence...

This attitude toward externalities โ€” not my problem, and in any case worth it in exchange for a small advantage โ€” follows the approximate logic of a spammer and often comes wrapped in the language of AI hustle culture. It's also understandable from the perspective of a job seeker who feels constantly thwarted by automated systems employers use that seem to treat seekers with similar indifference or contempt, or by platforms like LinkedIn and Indeed that, while nominally intended to connect two parties with shared interests (one needing specific services, the other offering them), can feel more like social-media-style black holes for engagement. It's an escalation that will likely be met with more escalation: countermeasures by job-listings platforms and hirers to prevent access by AI agents; more aggressive automated filtering; different hiring routines altogether, making it even harder to get through the door to a coveted interview. Mercenary (and slightly deceptive) automation tools like this, which are being pitched all over right now and already wreaking havoc in, for example, online dating, depend on two temporary circumstances to work, if they ever actually do: (1) that most other people don't have access to them, giving the user an edge and (2) that the people and parties on which they're used will tolerate and take no action against them. In other words, if you take their pitches at face value, they're pretty obviously doomed in the medium term, in the sense that they'll either be rejected by the systems they operate in or simply ruin them for everyone.
Taking stock of the first few years of mainstream AI deployment, though, raises an important question. What if that's sort of the point? Or at least a world worth thinking about in a more thorough, long-term way? Generative image and video tools, for example, have significantly degraded social-media platforms, allowing bad actors and regular people to fill them with slop, intensifying existing problems with spam and deceptive content while thwarting old solutions. And, hey, look at that: Suddenly, OpenAI and Meta are launching new social networks based on AI, on which posting generated content is the point, not a problem to be solved. Generative AI may be placing immense stress on educational institutions and worsening the already strained relationships between teachers and students, but wait โ€” every AI company is selling ed tech now.
4
15
48
John Herrman avatar
John HerrmanยทOct 6

it's only been a week but I'd bet a lot of early users are having experiences like this with Sora: manic, exploratory onboarding followed by something like a hangover. it's a pattern! nymag.com/intelligence...

Sora is presumably extremely expensive to run, hence OpenAl's use of a chained invite program to roll it out. In its early form, it brings to mind the early days of image generators like Midjourney, the video model for which Meta is now borrowing for Vibes. Like Sora, Midjourney in 2022, was a fascinating demo that was, for a few days, really fun to mess with for a lot of the same reasons: A vast majority of the images I've generated have been jokes โ€” most for friends, others between me and the bot. It's fun, for a while, to interrupt a chat about which mousetrap to buy by asking a supercomputer for a horrific rendering of a man stuck in a bed of glue or to respond to a shared Zillow link with a rendering of a "McMansion Pyramid of Giza..? ...I still use Midjourney this way, but the novelty has worn off, in no small part because the renderings have just gotten better โ€” less "strange and beautiful" than "competent and plausible." The bit has also gotten stale, and I've mapped the narrow boundaries of my artistic imagination. Playing with Sora is a similar experience: a destabilizing encounter with a strange and uncomfortable technology that will soon become ubiquitous but also rapidly and surprisingly banal. It also produces similar results: a bunch of generations that are interesting to you and your friends but look like slop to anyone else. The abundant glitches, like my avatar's tendency to include counting in all dialogue, help make the generations interesting. Many of the videos that are compelling beyond the context of their creation are interesting largely as specimens or artifacts โ€” that is, as examples of how a prompt ("sam altman mounted to the wall like a big mouth Billy bass, full body") gets translated into ... something. (As one friend noted, many of these videos become unwatchable if you can't see what the prompt was.)
Post image 2
1
0
6
John Herrman avatar
John HerrmanยทOct 3

part of the big theoretical "twitter do-over" is this inevitable jack-brained dynamic, too

1
0
17
View all posts on Bluesky