Sowmya Subramanian

And show me who you're with.

I'm with Quizlet as their CTO, head of engineering.

And what is Quizlet?

So Quizlet is a web mobile product and experience. It's a learning experience to help learners of all ages achieve their goals in the most effective way. So we have flashcards and study guides. And we just launched AI coach or conversational AI today.

So you can bring your own material. You can discover other people's content and just study.

So who's the target audience for it?

Our target audience is students, so college and high school students. It gets used quite a bit by also lifelong learners because it is a user-generated content platform. And the tools are available for anyone to use.

So you're just-- I mean, people are talking about how AI is completely effective in the way people are learning right now. You just launched a new product. How are you using AI to hopefully improve the learning experience?

Consumer products and learning products specifically. AI is having a huge impact on how people are learning right now. Do you view that as a net positive, net negative, net neutral? How are you feeling about that?

You know, I think-- I see it as net positive, if used correctly. So there's a lot of qualifiers in that statement. I do think AI is making learning of any skill, any content, democratic and more accessible by everybody universally. So what I might have had to go have internet access and go online or go to a school or find an expert to learn. Now, it's almost like Google search.

10x, 100x, easier. Giving access to anyone to be able to come in and say, "This is what I'm trying to figure out. Can you help me?" And the LLMs are there to help you figure that out. And it's not only a one-way seeking information. It's bi-directional. So you can engage with it. You can immerse in it. You can probe deeper. You can go sideways and explore. So you can do so many things in a more accessible, low-friction way.

The downside of it is, LLMs do hallucinate and they are as good as the content that they've been trained on. So you can end up with misinformation or even learning things that you think are accurate, but they're not. So how do you ground the data and how do you provide the guardrails so that when someone comes in from a learning intent, it is much more restrictive so you want to go for higher precision, even if it means that you're not able to cover it in that much breadth or depth. Can you lock it down enough? Well, we're all learning, right? So there are models to lock it down. So you can use MCP servers and get access to just the content and the repository that you wanted to give answers from and use as the knowledge source. And you can limit the knowledge source from which you want the responses to come. You can build out evals and grounding techniques to evaluate it with the ground truth and make sure that you are on the side of precision rather than trying to always give an answer, even if it means it's making up something. So there are ways in which to do it, and we've done this even pre-AI,

right? So previously I was in Google for 15 years, and as part of Google Search or YouTube or when we built the YouTube Kids product, we had to make sure that we are classifying content. We are doing content ratings. We are giving content that is engaging and enriching and appropriate to that kid or the user. So we're used to those kinds of things.

With everything changing as quickly as it is, upskilling, and the education that comes in that is incredibly important. You're a CTO, you're running a team of engineers.

How are you upskilling that team? Are you actively getting them? You need to use these tools?

Yeah. Or saying, pause. Yeah. You know, I think humans in general resist change. So if you force someone to do something, they probably are going to react saying, "I don't want to do it," even if it is interesting and exciting. So my approach is show the value and the why behind it and let people figure it out on their own. And then over time, you can do more and more as we all figure it out, because the technology is changing so, so fast.

And my ask from my team is just bring your curiosity, and it's okay to be skeptical. It's okay to be cynical about it, as long as you're looking at it from a place of curiosity, and you're looking at it to learn. And that approach has worked out really well. So initially,

in the recent organization,

I didn't require everyone to use AI. I just made it available. And I also started using it for my own onboarding purposes or for information that I would normally depend on my team on, and maybe they were busy or they didn't have access to it. I decided, let me write a quick GPT to go get that information and the JIRA boards or understanding the state of the code or the health of the system, things that it's more directionally I needed to get there. It needed to be super accurate. So by role modeling what is doable and possible, it then attracted people who were curious about that, and they were like, "Well, I want to do that. If you can do it, I can do it too." So now they started embracing it. And over a few months, we built out a community of AI adopters, and there was still a small group in the company, not the whole group. But with them then, now we have momentum, and they're learning from each other. They're seeing what's working, what's not working. So then I said, let's go back to the first principles.

And foundationally, computer science hasn't changed, right? So let's just look at what are the critical developer journeys. For each of these critical developer journeys, how do we--where can AI be used? And if you used it, what are the design patterns that you need to go after to use it? What are the tools? What are the guardrails? What are the checks and balances? So we started building it from there.

So as somebody who's got tremendous experience building high-performing technical teams, what advice would you give to somebody that's really building their team right now in this AI?

You know, I would--I think my advice would be the same as it has been through the whole time that I've been in the industry, I think. Look for people with curiosity or the growth mindset to learn, because we have to learn a lot on the fly. Two, never compromise on the fundamentals. The foundational aspect of things are still really, really important. So if you're doing a software engineering job, understanding the basics and the concepts and the building blocks of computer science are really important to be able to--whether with AI or without AI-- for you to build systems of scale and responsive and stable and extensible,

the basics are still the same. So the basics matter.

And third is team players. I think that's going to be really, really important, growingly so. Even as agents do a lot of our work, we are going to learn from each other and how we communicate, how we collaborate, how we articulate things, whether to a human or an agent or an agent form.

That communication skill that allows you to drive clarity and build partnerships and collaboration is going to make a big difference.

That's the most fascinating thing I think we're watching right now. Because historically, it feels like engineers are not the best communicators, arguably.

Yeah.

It depends. I think maybe they communicate in a different way, right? So I think everyone has to kind of just understand--I don't know. I don't know that engineers are not the best communicators. Maybe they're not the best human communicators, but they're very good at communicating logic and guardrails and corner cases and specifications. Are they the most ideal people to

move into the prompt engineer?

Yeah. Yeah. So it's a good question. Right now, prompt engineering is very natural language-driven, right? So if you can communicate, like write a beautiful spec or a nice design doc or design requirements, functional, nonfunctional, you're definitely at a better place than if you're just going to give like five rules to this thing and tell it to go do it, because you're then very prescriptive and you're not giving enough room for the exploration to happen.

So I think it's a skill people have to learn and adapt. I see it more as, you know, even as engineers, when you go from being a super junior engineer and you rise up as an IC, you still have to learn how to delegate, how do you do code reviews, how do you give input to others and mentor and bring them along. And think of the LLM as that coding assistant or partner with whom you're prompting it to do interesting things with you and amplify the work that you're doing.

We're all going to kind of figure that out together. Yeah. Like, who's the future of a junior engineer? Are they the best? Is the best junior engineer someone who's got the fundamentals down and is the best buddy with the agents? Or it's going to be fascinating to know what that looks like.

I think so. And, you know, for me, it's less junior or senior engineers, honestly. Like, I would hire out of college any day, you know. I think it's the attitude that matters. Like, are you there to--it's not going to work the first time you try it. Maybe it's not going to work the fifth time you try it. But do you have the perseverance to continue to figure out, like, what is working and what is not working? And when it's not working, how do I adjust it? And when it is working, how do I make it a multiplier for me? And if you have that approach to take,

I think your seniority really doesn't matter. I do think what matters is knowing when things are veering off,

because many LLMs are super confident, right? So they'll come back and make it look like everything is amazing. So you need to know, and you need to have those battle scars of building and scaling systems to know, you know, are the security--have you thought about security? Have we looked at the corner cases and what do those look like? Or in scale or at this kind of performance, what is going to happen?

That's perfect. Okay. And also really interesting. You need a framework for failure.

Yeah, I don't know. But the answer is okay.

No, no, absolutely. A framework for failure and the ability to call bullshit are kind of this core of what makes it good in-- Yeah.

Yeah, I think so. And I think that was always true,

even without the LLMs.