Local news, unleashed: How Inside Arlington is using AI to cover town government

On October 25, 2023, the AI Literacy Lab and the Northeastern School of Journalism hosted a lunch with Winston Chen and David Trilling, the founders and editors of Inside Arlington, an experimental website that offers AI-generated summaries of local government meetings in Arlington, Massachusetts. 

Chen is a technology entrepreneur who sold a speech-to-text technology. Trilling is a former foreign correspondent who ran a newswire about Asian news. What follows is an edited and condensed transcript of the conversation about their new project and its implications.

Joanna Weiss, executive director, AI Literacy Lab: How and why did you start this website?

Winston Chen: It started with a dog. One morning I went walking the dog and there was a lot of uproar in the dog park. Through the pandemic, leash laws were not enforced very much.. But all of a sudden, people were getting fined. All the dog owners were up in arms. I said, “Let’s go to the Parks and Recreation Commission,” because I know that’s where the rules get made. And then everybody’s like, “Oh” — just a collective groan. “Two and a half hours. I don’t have time for that.” 

When David and I started talking about this, I suddenly realized that here’s an area where AI can play a big role. I spent a few years at MIT Media Lab, and one of the hot topics back then was the destruction of local news. We thought, the problem with local news is largely economics. So what if we can provide any local news outfit with a set of tools? Not just technology, but also human resources, legal, finance, various types of support so that the person who’s responsible for a local news outlet just has to worry about the content. “Local news in a box” was the idea that was born. We put it aside, we got busy with other stuff, and now with all of these thoughts [about AI], that idea has bubbled up again.

Joanna Weiss: Walk us through the site, how you designed it and how it operates. It’s very simple and almost entirely automated. 

Winston Chen: We have a local cable access network. They record a lot of meetings in town, then post them on YouTube. So every night we have a process that goes out and finds if are there any new recorded meetings. If there’s any new content, it downloads the audio from YouTube. Then we convert that audio into a transcript. We use a third party service for that. 

David Trilling: We can feed it some words that we know it’s going to struggle with. Names. We’ve given a list of names in town:  Don’t mess this up.

Winston Chen: We also feed it the content from the agenda, which we automatically grab. Once we have that text, we do the second stage of our process, which is sending it to a large language model. The best one in terms of output is ChatGPT, which [uses] is a base technology called GPT-4. We give very specific instructions on how to summarize it. 

David Trilling: We have been constantly tweaking the instructions as we’ve gone along.

Joanna Weiss: What did you start with and how did you end up perfecting the prompt?

David Trilling: Bringing in the agendas was a big help. Just telling it: Ignore this, focus on this, focus on different speakers. Stop telling us what time the meeting adjourned that, but focus on votes. Tell us what the vote was. 

Winston Chen: In the beginning, we discussed this extensively: Should we ask the AI to write a story about the meeting in the style of a journalist, or should we ask the AI to write something more boring? And we made a conscious decision to make it boring. The AI doesn’t have the context to make a good judgment. Of the five topics discussed in the meeting, which one is the most interesting to the reader? So we give the AI instructions to write a list of topics discussed. And for each topic, talk about what decisions are made. It’s designed for someone to scan from top to bottom, say, “Oh, I don’t care about this, I don’t care about that, oh, here it is, I care about that.”

David Trilling: Also, we haven’t run into hallucination problems, but if we were to make it more interesting, I think the fear is that there may be more likelihood of hallucinating.

Joanna Weiss: Once this summary is generated, do you look it over before you push “publish”?

David Trilling: Yes. I get an email saying there’s a draft post in WordPress, and I go into WordPress and tinker with a few things. I try to check people’s names. I give it a pretty quick read, make sure, from my knowledge of what’s going on around town, things don’t sound a little too out there. Occasionally I’ll add a couple of words of context and push publish.

Joanna Weiss: Do you get a sense of how many people are reading it?

David Trilling: We never intended this site to light Arlington on fire. This is our pilot, our experiment. We’ve attracted a few hundred subscribers and I think every post gets a few hundred hits, some posts more than others.

Winston Chen: A lot of town officials are subscribers to the newsletter. That makes sense because they really care about what happens in these meetings, and they — just like everybody else — don’t have time to sit through hours of meetings. There’s a committee meeting almost every night of the week.

David Trilling: And no one has time to go to them all.

Joanna Weiss: We’ve had lots of conversations with journalists who are very concerned about a future where AI is unleashed to write stories. Gannett has been experimenting with using AI to write local sports stories, and sometimes it comes out with stuff that is either laughable or offensive. How would you persuade reporters that this is a tool and not a replacement?

David Trilling: I hope journalists have an open mind. This is a tool that can help. The fact is, it’s an industry with very little money and very few jobs, and if this can make you more productive, it’s going to help your outlet thrive. I don’t think it should replace humans. I think it should make you more productive. But the fact is, we are confronting a trend where humans are getting replaced in media, and whether it’s AI or it’s just by some C-suite executives looking at the bottom line, it’s happening. 

Michael Workman, creative director, AI Literacy Lab: I’m curious what you’ve learned about Arlington government, what’s surprised you. You probably have seen more of the Arlington government than anybody. 

David Trilling: Issues get heated, for sure. To fact-check,  I’ve attended a bunch of meetings. Zoning board meetings can be very heated and there’s a lot of nitty gritty going on that our local representatives are doing. I had no idea and I’m really appreciative of their work.

Joanna Weiss: That does raise a future-of-journalism question: Is there something you miss by not being present at the meeting? Are there things you can get as a journalist, when you’re physically present at the meeting, that AI can’t replace? And how do you strike that balance?

David Trilling: It won’t get tone, of course. And it doesn’t get context. A reporter should be looking at: What are the Facebook discussions going on, what are people arguing about? There’s a Listserv in Arlington with a lot of people arguing all the time, and it doesn’t get any of that material. So that kind of context, we need journalists to be on top of, to write the best possible report about a particular government decision.

Winston Chen: I guess I’m a little more optimistic about the future. Right now, we take the audio of the meeting, which is already missing some stuff.  You lose facial expression and then once you transcribe audio into text, you lose more. Open AI just came out with [a program] that can process images. I just took an image and asked questions about the image. It’s uncanny how good of a job it does. So I wouldn’t be surprised if, a few years from now, you can take video and summarize that, and the AI can catch all the nuances in context.

John Wihbey, Northeastern professor of journalism: You guys were talking about the “local news in a box” concept. What are your plans for the future? Are you trying to get funding so you can commercialize this?

Winston Chen: We have a not-for-profit entity called Nano Media. This is sort of a test case. The idea is that the set of tools and practices we have developed, we’d like to spread this to other towns. There are two scenarios. One is that there is no news outlet in that town; we help someone or a group to start a brand-new site. Another is there’s already an established site in the town and we want to supplement their content. We can help them understand what happened in these meetings. And then if the reporter finds something interesting in the digest, they can go and do a human-based report out of that

Another area we have ideas about is: How do you collect information from different sources, compile it and have a more automated way to tell a human reporter what’s going on in town?  Just imagine if you are able to see what’s going on Twitter, what’s going on Facebook, what’s going on in a listserv, what’s going on in various parts of the media landscape of town, and then pulled that all together. These are the potential topics for a journalist to go and explore and write about.

Rahul Bhargava, Northeastern professor of journalism: The data streams in a community are very unevenly accessible. To the Latino, it’s all going to be on WhatsApp in the US, so you can’t get that data stream without having access. If you want middle class white, it’s probably on Facebook or some other platform that has an open API. 

Winston Chen: When I was at the Media Lab, I worked with a political scientist who wrote a book about Wisconsin politics. She gathered information about what goes on in these small towns by just going to, sometimes it’s a gas station, sometimes it’s a diner. She would sit down: “Can I join you guys?” And that’s where information exchange is happening. AI isn’t going to help there. Depending on the type of community you’re dealing with, the benefits are very uneven.

Dan Kennedy, Northeastern professor of journalism: I think a lot of us look at local news as part of an ongoing conversation that a community is having among themselves, and the process is as important as the result. The published articles, it’s a way of building civic engagement. And I am wondering how increased automation contributes to civic engagement. It seems to me that you could make an argument that it detracts from civic engagement.

Winston Chen: In the end, whether it’s a piece of paper or it’s a website, it’s media, right. It’s a media in which ultimately humans exchange ideas. So we do not advocate for a world where AI drives the flow of information. We believe that humans should always be in the driver’s seat. AI is helping the information get to the right person and helping to make information more digestible. For example, you can post a two hour meeting on YouTube, or you can read a much shorter digest. You can say, well, the AI, by converting the two-, three-hour video into bullet points, is increasing civic engagement, because now you made what’s going on in the government accessible to more people. 

David Trilling: It’s sort of like journalists from a slightly earlier generation who maybe ignored social media. You can ignore it and keep reporting and maybe writing great stories. But social media, for all its flaws and the horrible things it’s done to civic life, it also is a tool. If you started out ignoring that, now you’d be kind of behind and having missed out on that tool. It’s changing so fast all the time that I think, especially as a young journalist, you would be at a real disadvantage if you didn’t try to understand how it works.