Episode 3: Jorjeta Jetcheva

[Music]

Vincent del Casino: Hi there, and welcome to the Accidental Geographer. My name is Vincent Del Casino. I'm the host of this podcast, as well as the provost and senior vice president here at San José State University. Today on the podcast, I have Dr. Jorjeta Jetcheva, a professor in computer engineering here at San José State. We're gonna have an amazing conversation across a number of different topics related to artificial intelligence, the future of human computer interactions, as well robotics and how they're gonna play in our everyday lives. So come on board. This is going to be a great discussion. Well, thanks for being here. I really appreciate it.

Jorjeta Jetcheva: Thank you so much for the invitation. It's a privilege to be here.

Vincent del Casino: You know, I always love to start the conversation with sort of how you got here and into the academy because you have an interesting career. You've obviously studied, you've done research, but you've also worked in industry and now you're at San José State. Kind of what brought you here and what parts of that trajectory have you continued to evolve as you've been here?

Jorjeta Jetcheva: This is a great question, and the answer could be very, very long. I'll try to be brief, though. So I, you know, I always liked helping others understand concepts. So I mean, as recently as, or as far away as elementary school. And, but I never thought of this as a career, certainly the teaching aspects. Then I had the privilege of tutoring a high school student in math. She was in ninth grade, and we just sat together. She was doing her homework. Her mother told me she's on the verge of failing math. I sat with her when she ran into a problem. We worked it out. I didn't see anything wrong with her approach to math or her ability to grasp concepts. By the end of the semester, she was getting B pluses and A minuses. And she told me, well, you know, my teacher said, oh, I don't know what your tutor is doing. But she said, we're going to start trigonometry, and I'm going to get you there. So the teacher also somehow felt this was unusual. But I didn't do anything special. And this is what motivates me to this day, is that we have students who have the abilities, but for whatever reason, have fallen through the cracks. And it's such a wasted potential. And so unfair for them to live with the thought. That they cannot handle math or science or what have you when that's very, very far from the truth. Still, this didn't make me think of teaching. I had a lot of encouragement in college to pursue research. Professor Claude Fenema, my undergrad advisor, was amazing. He always talked to me about research. I started doing research when I was a sophomore. I did an undergraduate thesis as well. And then I applied for jobs and also for graduate school. And I was thinking, well, I probably don't want to spend too long in graduate school, but you get sucked in. It was an amazing experience. But even through that time, the narrative was research is the cool job to be doing. Even if you're at a university, it was more about research than teaching was not even somehow in the picture. Um, so, I did apply for some academic jobs at the time. The academic job market was very bad at that time. And so I went into a startup, which was what I had said I would never do. But the job was interesting, and I do that when something is exciting, I will go for it even if, you know, it wasn't what I was planning to do. And then I started thinking about research again, and I wanted to get back into research. I finally managed to get into producer research, And then I started thinking about academia again. So I applied for a position at Santa Clara University. I taught a couple of classes there, but that didn't work out. And then later, though, as time went on, I started feeling like, you know what, actually the thing I'm interested in is mentoring and teaching as opposed to research. Right. So then I thought, well, what is the place that I can go where I can fully explore that and fully contribute in that way? And so. San José State was right here. I coincidentally was invited to the Women in Engineering Conference. I was at Accenture at the time. And the rest is history as we speak.

Vincent del Casino: That's awesome. So when you did your PhD at Carnegie Mellon, right? You did it at Carnegie Melon. What was the focus of that work? And what did you first do in that startup? You know, what got you into the startup world, which is also really interesting. I imagine you have students who are like, I want to be in the startup world and you have background in that directly.

Jorjeta Jetcheva: Yeah. So my work was focused on self-organizing wireless networks. So at that time, actually in 1997, we had kind of the pre-Wi-Fi, the first kind of version of Wi-Fi. We had, I think it was one megabit per second. And we were doing, because we wanted to have networks that self-organized, so they find how to pass packets among the nodes automatically. We had rented six cars, and we were driving them in a parking lot. And each of them had a wireless router on them, and they would adapt the routing paths based on who they were near to. And then there was also kind of a main base station and how to connect to the base station through other nodes. So we were doing this kind of work. And at that time, there was a lot of uh interest in broadband communication on college campuses, company campuses, within cities, some municipal areas because we didn't have the cell phone infrastructure to provide high bandwidth communications. So Firetide was a startup who wanted to deploy these kind of networks in real applications including, let's say, to provide Cupertino with broadband access. So no matter where you go, you have Wi-Fi, which of course now is the case, but this was 2004. So they were looking for someone, they had put something together, but they realized it's not quite so easy to just take some open source code and get it to work in this setting. So they had met my advisor at a conference and had contacted me. Because they wanted someone who understands how routing works, how adaptive routing works in this kind of network, how to construct the network architecture and protocol so that this works in a real application. So they left me a couple of messages which I didn't respond to because, you know, I thought, well, this is a startup, I don't wanna go to a startup. I was an international student. One of my concerns was, you know what happens when the startup folds and you know, I have to go back to my home country if I can't find a job within some number of days. And I know students face that question a lot. You know, what kind of risks can they take when they're international students? But I joined the company. I was there for about four and a half years, at which point the 2008 recession happened. And we had a massive layoff. But it was great. I worked with engineering teams to create network architectures to figure out how packets should be routed between our network and other networks. We had some deployments with police departments because they wanted to have their different vehicles interconnected with each other. So when they're driving to a location, they can have maybe video and other situational awareness, essentially. That they couldn't with existing infrastructure. So it was a very, very cool experience.

Vincent del Casino: So when you, so going back to 97 to 2004 to 2008 and beyond, obviously the ideas of artificial intelligence, like the notion that AI could be out there, that there could be intelligent agents, that concept has been around a long time, but you're actually on the ground starting to see it activate because you were working, I think, in. At the intersection of these software questions, but also robotics and other things like that as well. So both these things working simultaneously and the idea that you could network all these objects together wirelessly, that's less than 30 years old as a concept to be able to produce the technology to do it. So how did you think about that? And back in 2004 and so forth, did you see us where we are now? And if so, what were the things that slowed us down? Or what were things that you worked on that helped us move it forward might be the better way to put it.

Jorjeta Jetcheva: So this is a great question. And actually, my answer is a little bit different, I think, from where the question is going, which is the first time I started work in AI was as a sophomore in college. So this was 1994. And my undergraduate advisor, Professor Fenema, had a robot and was focusing on computer vision. So I spent a semester doing that, and I felt that the technology was so behind, that there was so little you could do with it, that I told him, you know what? I just, I don't want to work on this anymore. I want to do something else. And I went and I came up with an idea to still use the computer vision algorithms, but focus on How do we distribute them so that they run across a network of workstations? At the time, there was a popular concept that Berkeley was doing a lot of research on, where there's a lot unused processing power across people's workstation. How can we send processes there to use it to speed up computation? So yes, at the time it seemed like this was really so painfully just rudimentary. You couldn't do anything real. What we were doing was we were looking at images and trying to group together pixels, so dots on the image, that had a similar color to try to differentiate kind of objects from each other and create boundaries between objects. So when I went to grad school, I took an AI class, it was required, but things still seemed very painfully just rudimentary. And so I focused on wireless networks that you know, seemed very cool. Somehow the idea of sending things, you know. You know, communication that you can't see was interesting. So yeah, in 2004, it still didn't seem like this was an interesting area. So come 2008, you now, firetide, the startup had a layoff. And suddenly I was in a position where I had a lot of time to look at what are, what kind of jobs are out there. Especially in a research context. And I saw at that time that the number of jobs that focused on using data to extract insights was really large, I mean, disproportionately large relative to looking for wireless or networking jobs in research, that was a very small piece of that. So then I said, OK, and this seemed very interesting to me also, just extracting insights from data. So I started looking for how to transition to that area. But I couldn't at the time, because I had no background in it. And but what was, at that time, growing was the smart grid sector. And the smart good sector had a networking problem. They had millions of smart meters. And they wanted to interconnect them dynamically. So let's say you have a network of a million smart meters and you want to collect data from them every 15 minutes and the data needs to be sent to some kind of centralized office. To do that, the range of these meters would be pretty small usually. They would have to send their updates hop by hop to each other, sometimes over 10 to 15 hops to get to the, you know. Front office or whatever, a centralized collector.

Vincent del Casino: What kind of meters are we talking about like is this in a factory or is it out in the world like Gas meters. I'm just curious like it or is any in above any of these things?

Jorjeta Jetcheva: So the main focus was electric meters. Got it. And the idea is, if we can measure usage in close to real time, we can predict electricity demand. And then we can balance production and supply and demand of electricity more accurately. And that gave rise to other applications. But initially, my focus was, how do we interconnect these meters? Because also the thing is when. There are obstacles in the path, so let's say there's a truck, suddenly the signals are bouncing around, you'll have to find a new path, part of that path is broken. So this network is very dynamic, but very large, and these meters have very limited compute capabilities because initially they were built to just measure electricity to charge people for their usage, but now we're trying to retrofit a measurement infrastructure on top of that. Um, but, uh, You know, so I started doing that and then it turned out that research labs wanted to get into the smart grid domain, but there were no PhDs with that background. So, you know, I was, you know fortunate to get in a position where I could do smart grid related essentially data mining, and started to do machine learning with that as well at Fujitsu Research.

Vincent del Casino: That's really, really interesting. And so you ended up working on wireless networks themselves. Then even though you didn't have that background, you ended it up there, which is the backbone of a lot of what we do. I mean, it's amazing to think even five, six years ago at a university, I would often dock a computer and have an ethernet cable. We don't do any of that anymore. So you're building there. And moving to the smart grid. And emerging out of that is the, quote, IoT, the internet of things kind of phenomena. It's interesting. People don't talk about it much today. But it was a very popular moment. And that was a lot of what you were working on at Fujitsu, right, was the question of, could we link all these objects together in that network so that literally I could control my home from my phone, right? It's that starts at talk through a little bit about the emergence of the internet of things and how some of the conversations you were having started to intersect with human computer interaction research and how this might be applied beyond industry or things like that. What started to come about? Because you were there for about six years doing that work, I think, right?

Jorjeta Jetcheva: Yeah. Yeah. So, um, towards the end, I was more focused on kind of human computer interaction and robotics and, um virtual assistance, but, uh, the first four years, certainly that was my focus. Um, there were two areas of IOT. One was kind of industrial or, um enterprise devices like, um smart meters or anything you want to deploy out in the world, like, uh you where you can control lights or traffic lights or just regular lighting. And one of the goals behind some of these capabilities was for the electricity company to again, be able to better manage supply and demand. The other element of this, which was more focused on home automation is where you have your refrigerator, your washer and dryer, your lights, also having wireless capabilities. And of course, there's a user aspect of being able to control these remotely. Let's say that you've gone on a trip, you realize that your heater is still on and it's gonna be on for a whole week at, I don't know, 75 degrees and you're wasting a lot of energy, et cetera. And then there's also, again, the utility or electric utility angle, where at times of high demand during the summer, let's say, you know, and this is one of the types of technologies we worked on, was demand response. So at times high demand, if you have given permission to, lets say, PG&E, they can temporarily turn down your AC or other potentially devices. ACs were the most inter... Connected so that was a natural choice but certainly there are things like water heater that we don't even think of but it's on all the time you know anytime the water temperature drops you know so anything that was using electricity so so again because the alternative is you have a blackout right and this way if it's possible to you know first of all you're tracking usage of the different homes. To the point of we had projects on disaggregation, where you're trying from the electricity usage from the signal to figure out, is the person using their water heater or AC, or how much each device is using in order to avoid blackouts by controlling devices judiciously, and not just stopping the device for 30 minutes, but things like, well, can we do real-time demand response? So I. Turn down your IC for, I don't know, one minute. So maybe things like this where you're not impacted, but overall you're aggregating a large amount of electricity usage.

Vincent del Casino: Well, now I finally understand how PG&E can know how often I run my washing. So, in that, there's some interesting questions that emerge, you know, around IoT that I want to kind of touch on before we get into some of the other things, because obviously, there is machine learning algorithms, there is the basis of what we now kind of lump as artificial intelligence. And that term gets so broadly used now it probably encompasses too much. The interesting thing, we started to get smart light bulbs, we started get all these things. And one of the things that was interesting is now we've opened up our homes to these networks and the potential for hackability and things like that. And I know early on people were concerned about that. How does the work you were doing, maybe you weren't on the cybersecurity side, but that conversation had to be going on simultaneously is how do you protect these data flows and then protect. The inputs in these homes, because now all of a sudden, everything is plugged in literally at a very high level.

Jorjeta Jetcheva: Yeah, absolutely. And I did, at that time, focus on some aspects of cybersecurity, but they were more on the network side, not on the device side. But certainly, there were big stories, even years ago, of just refrigerators being used to propagate messages as part of a cyber attack. Um, and one of the things I noticed, um, because I was very excited about all of this at the time, I got a washer and a dryer that had a Wi-Fi capability, and a refrigerator that had Wi-fi capability. All of these devices usually had a WiFi card that was only capable of an older version of the WiFi security protocol. So now to connect it to your network, either you have to know a lot and set up a separate network, a separate WiFi. Network for just these devices and have lower security for them, and still they're hackable, but at least your entire network has not dropped down its security level. Or most people may have just said, you know what, I'm just going to select a much lower security grade in order to connect these devices. So because appliances have a long lifespan, you end up compromising security almost by default because if you're unable to upgrade them somehow to these new security protocols.

Vincent del Casino: So is that where you got interested in encrypted traffic? I know you've done some collaborative work on that. So encryption has got to be important in all of this as well and more and more. I mean, at the end of the day, part of the big reason for quantum computing, people think it's about speed. It is about speed, but it's also about our ability to do such large numbers that we can really protect data, right? Because we need a certain level of computing to get to a level of complexity. So it's interesting that you point out the smart device, is it upgradable over time? Does encryption play into that at all or is that what got you interested in encryption and kind of what did you find from this work?

Jorjeta Jetcheva: Yeah, so the encryption level was weaker for these home devices, certainly in my experience. I have always been interested in cybersecurity. This was more incidental in that I had a colleague at Fujitsu Labs. He was working on security, and he started a collaboration discussion with some folks at UIUC, and I joined in. When you're in a particular environment, there are a lot of issues that. Interrelated issues that you end up having access to. And I really enjoyed working with these folks, so I got deeper into that area. The focus of that project was more on the smart metering infrastructure monitoring side. So trying to catch attacks against the smart metered infrastructure, because again, smart meters are much more, much less capable devices, and there are many of them out there. And in security, in the past, we used to say, if you have physical access to a device, to a machine, you could do whatever you want. So it's really hard to protect a device when someone has physical access it. Well, these smart meters are in people's houses, and they're on the street. I mean, you can walk into the yard and, you know. So the idea was how to protect this kind of metering infrastructure. And on top of that, how to do it without introducing devices that can decrypt the traffic. Because the traffic within the smart meeting network is encrypted, if you want to do security checks that require that you sniff the packets and decryp them, then your device becomes very attractive for being cyber hacked. So the idea was, can we see patterns in the communication? That indicate that there's an attack without having to decrypt the packet. So use information from packet headers that is never encrypted, because routers have to know where to send the packet, so there's some information that is not encrypted. So we did some of that work, trying to see how much can we extract, including using machine learning, about the communication patterns to detect when there's a difference. And part of that also was to to detect, to identify what kinds of packets are being sent. So to the point of is a particular operating system, let's say is it a Mac or a Windows machine that is sending some of these packets, there's some signatures or were at the time. I haven't looked at how that has evolved. Again, is a form of disaggregation where you're trying to separate what is in the traffic stream. Again, when this traffic stream is encrypted.

Vincent del Casino: So when you got to Fujitsu and started, you started working on intelligent assistants and other things like that as well. And again, did that intersect with your interest in robotics? How did all that play together? And how has that led to some of your interests? Cause you've now worked on some large language model questions and other like that, which really sits at the heart of a lot of the generative AI conversation. And what's going on. So how do all these pieces sort of fit together for you intellectually?

Jorjeta Jetcheva: So, you know, I, well, again, some of these things, you have some interests in mind and then some opportunities arise that you didn't even realize, you might be possible. So, it's almost that things come together coincidentally. At the same time, when you kind of look at how things came about, you realize that, you know you kind drove a momentum in a particular direction. So, I moved from the Smart Grid group. To what I thought was gonna be the health group at Fujitsu Research. I was very interested in health. So for a long time I was thinking how to get more into this area. And it was AI for health essentially. But then as soon as I moved, research labs kind of changed priorities sometimes very drastically every few years. So the focus of the group changed. So we weren't doing any health. But one of the projects was around intelligent assistance. So how do you, if you have HR questions, how do essentially build a chat bot to answer these questions? Of course, the technology at the time that was available was very rudimentary. So, but we started doing some of this. So, and the first part was, you know, let's collect a bunch of questions that people might have, let's say about sick leave or vacation. And then just create kind of a set of unique questions that we wanna use as a baseline. And it turned out because we use crowdsourcing, so we use Amazon Mechanical Turk, we had people submit questions, that the technology at the time was not even good at helping us identify duplicate questions. So suddenly you have thousands of questions and you're very excited and then you realize there are many duplicates and as a human, And I cannot just go and you know, easily, you know de-duplicate them. So we started doing various clustering, so try to cluster similar questions together to the extent possible. So that took a lot more time than we expected. And it's really interesting because you start to get into things like, well, you now, there are these few questions that let's say the algorithm has clustered together, let's look at them manually and see do we agree with this. And there were, you know, for this part of it, there were three of us kind of trying to decide, are these duplicates or is this different enough that it should be a different question? And we had disagreements. So, this is another problem with, you now, language and with human judgment. We, you, know, what might seem obvious to one of us, you know is not obvious to others. So how do you reach agreement and what conclusions do you draw? So this is kind of where this project was at the time that I left Fujitsu Research. But the robotics project was completely different. So Fujitsu research started a program at that time, which was unusual, and said, we'll give you some money, propose an idea, pitch an idea. And if you win, or there were several projects selected, we'll give you some money to work on it and create a prototype. So, I really sat down and thought, well, what am I excited about? And at that time, I was excited about finding ways to kind of facilitate everyday life by using, in this case, a mobile robot to go around the house and keep track of where things are. When things have to be replenished, like you run out of toothpaste or soap. It just, it felt like there's so many things you have to keep track of. And one of the first things I did was I conducted a survey. I asked a number of colleagues at the lab, can you write down how many of these consumables you have in the house? You know, it separates food from other things. You know I mean, just even toothbrushes Other things you have to periodically... So, and I don't remember the numbers, but they were, the average was over 100. So we don't realize how many things we have in the house that we have to keep track of, and that run out at the wrong time. And now if you order too many of them, you don't have the space to have, you know, just boxes and boxes of stuff, everything. And they also go bad, they get lost because you bought too many, you've shoved them in the corner, you lost track of it. So this was kind of the idea. And so I was able to hire a couple of people to help with the development. And I hired a San José State student to create a video at the end. This was very unusual in the lab. Nobody was creating videos, but I, but I felt that, you know, this is an important part of the process for some reason. And the video was, you now, amazing. The student was amazing.

Vincent del Casino: So can I pull on a couple of the threads there? So going back to the AI conversation for a second and the artificial intelligence and the chat bots, there's something really, there's a couple really powerful things you said there. This whole conversation of training models, like how we train them. We're now gotten to a place where machines can start to train machines at some level, but that early work, that disagreement, which still exists, right? It still exists now. But that disagreement in that room shows how powerfully connected human intelligence is to all these systems. And in fact, that training, what we see as humans is, for lack of a better word, if I had enough time, I could simply look at things and start to cluster them. For a machine and an algorithm, that's actually incredibly hard. So how did this negotiation between humans turn into training for some of the models that you were building earlier?

Jorjeta Jetcheva: Um, so, you know, I mean, at that time you basically have to make some decisions and they're not perfect. And, and even now, so you know because we're the early stages of automation still, we have the understanding that the systems are not perfect, but they are helpful to some extent. So we just, we, on one hand, just understand that there are limitations. The other element of this is, and I got to work on HR use cases at Accenture, and we actually started deploying virtual assistants for HR. We had deployed a pilot internally, and then we deployed the actual virtual assistant at UPS, so very large companies. And what we started doing was very heavily relying on the domain experts. So. We had to have really a very good discussion with the HR team. How do they separate the different topics within HR and then the different questions? What is considered? Should a question be part of payroll versus something else? So domain expertise is extremely important always. And as technologists, we always should work very closely with the domain experts. So this is how we try to compensate for some of our disagreements. Let's just go to the domain experts. I don't need to decide if this question falls into this category or the other. So this one of the ways to do that.

Vincent del Casino: So you also said just now, which is interesting, that automation is still in its early stages. Because there's so much attention at this point on what machine learning can do, the advances, its ability to obviously digest information and look at it at speeds that are well beyond the human ability to do that at this. And yet... I've had several folks in the conversation here and there's a sort of, we're still in the nascent stages. So when you say that we're at that, we're in an early stage of automation, can you flesh that out for me about what that means?

Jorjeta Jetcheva: Sure, sure. So, you know, it's all fun and games when we are kind of using the technology individually, but let's say you are trying to deploy this to a, let's a virtual assistant for HR, to a company that has hundreds of thousands of employees across the world. And they're asking you the question of, is your chat bot or virtual assistant or agent providing correct information 100% of the time. If you can't say that, then they start to be very hesitant. So what are the guardrails you're gonna put in place to ensure that wrong information is not provided? And so there are many situations where it's not okay to make mistakes, even 5% of that time. And these are models that we have very little control over and very little kind of theoretical ability to analyze. So we don't know in a real use case where hundreds of thousands of people are using it for a particular use case, how many times and what level of false responses it might provide. It's not even about being false. A lot of times it's, you know. People ask very sensitive questions of HR. So someone might say, you know, my spouse has a very serious disease that I need to take time off. Or I'm going through a divorce, I need do X, Y, Z. If the agent or a virtual assistant responds insensitively, that is also problematic. So this is the main point. In real use cases, you cannot afford to be wrong even a small percentage of the time.

Vincent del Casino: So we're at a stage where things are, those machine algorithms are still learning. They're very young, I guess, might be the way to think about it. And the other side of it is, when does the algorithm cut itself off and go, you still need to talk to another person. Like we've gone beyond my abilities here. And that's in part because I think, I'm gonna say something and then you can correct me. The algorithms can still go, they go pretty deep, but they still have a hard time going wide. Like, you know, I think we're getting in a place where reasoning is becoming more common, but they're often working within confined data sets. They're not grabbing from everything. And humans can go this, you now, side to side with ideas, right? But a computer has to still dig in pretty deep. It's not as simple to make. Inferences from across all these different sets of information, and in some cases we don't even want them to look over there, right? We want them stay contained. Is that sort of an accurate representation of what's going on right now?

Jorjeta Jetcheva: Um, I think the, uh, maybe an additional layer on top of this is that we want to kind of have almost like a weighting factor on different sources of information. And that we cannot control very well. So even if we tell the model, well, this is very important, focus on it more or, uh. Override use this information to override previous information you've seen because we don't know what they're trained on really It's really hard to enforce that But you know, there's some very interesting work from Entropic They published a paper recently About their tracking how a large language models are kind of you know What steps they're going through thinking through things? And they've seen kind of a, you know, Tennessee to, you know, intentionally mislead. So if they feel that, you know, so if you give them a prompt to follow a particular directive, they might misuse that directive to lie, because they feel they're protecting the directive. So it's really just a Waccamaw kind of issue. So that's the problem. And of course, these models are super useful. It's just that in real applications, you have to build so much scaffolding around them that it really takes me back to what we used to do prior to LLMs. What they're very helpful in is the duplication problem I mentioned earlier. So now if a user is asking the question, at least I can figure out which of the questions I'm expecting does their question map to. Without having to do some very, you know, basic clustering and error prone clustering.

Vincent del Casino: Right, right. In the large language models that now are coming.

Jorjeta Jetcheva: Yeah, yeah, exactly. So if we use them in very strategic ways and do a lot of guardrails, I mean, so now a lot the guardrailes that they use are based on the question that you ask. But it seems that the answers they provide are not really vetted. So I had some students in my NLP class, my graduate NLP.

Vincent del Casino: Which is natural language processing.

Jorjeta Jetcheva: Present papers, yesterday I've assigned them a number of papers, and they've been presenting kind of more advanced technologies, and now I forgot where I was going with this. So they had a jailbreak, oh yeah, so they had paper where, so you're not, this is paper, I think, from Entropic again. So models catch questions about, you know, show me how to build a bomb. So they're now increasingly, when you know you ask a question that's inappropriate that someone has flagged, they will not respond. They will say, sorry, I am not providing this kind of information. But now instead, what some researchers are doing is they creating kind of an ASCII kind of image of some of the words. So show me have to build and then bomb is just, let's say, tease, but they form. The word, the word bomb. Wow. So then the response is, oh, this is how you build a bomb. So the answers are not really vetted, it seems. And again, when this kind of work comes out, people start fixing or, you know, introducing additional guardrails, but there's still many, many ways to get, you know inappropriate information.

Vincent del Casino: Yeah, so it and and still a lot of inaccuracy depending on how wide you cast your net. So you bring up the students I'd love to transition to there because obviously you're here now because of what you said at the beginning, you're excited to work with students and San José State students are incredible. They come from these amazing backgrounds and you've got National Science Foundation money now to work with lower income students and others to get them excited about what you're doing and getting them integrated. Tell me a little bit about that work and then tell me about how you're engaging students because they're doing great. They come out of your program, other programs here, they're getting really great jobs and people love working with San José State students. So how does that all come together for you and what has the classroom experience been like? Has it lived up to your expectation of what you wanted and what are the kinds of things you're working on in your lab with students? That was a lot. But I'm excited. I'm excited.

Jorjeta Jetcheva: Yeah, you know, and it's a great question. I think the students are amazing. They're so excited about learning. They come up with their own project ideas. So, and I really like that. So I tell them I can, you now, make a suggestion for a project that you can do, whether, you it's their senior capstone project or a master's thesis, but they always have much more interesting ideas. And the part that bothers me is that I wish I could work with more students. At once. So I'm working on how to scale, how to skill potentially advising, you know, if we figure that out that would be great because we, you, there are students who want to be involved in research that, you now, are not necessarily going to have the chance. And the classroom experience is amazing as well. I mean, I wish I could work with, again, teach more classes. Uh, but it is hard because I, you know, especially with classes related to machine learning and natural language processing, things change so quickly that even a single class has to be revised drastically because you know these students, they, they know what was announced yesterday and they're asking about it.

Vincent del Casino: They're asking about it. That's right. They're they listen to the Nvidia presentation at GTC and they know how fast the new super computer is going to be right.

Jorjeta Jetcheva: Oh, yeah, exactly. And, you know, they, they. They have a lot of questions. They want to know the latest and greatest and, you know, someone released a new model yesterday and their source code out and you know.

Vincent del Casino: Yeah, it must be challenging just to keep up with that academically for yourself. And now you've got all these students and you're right. Everything is very much in our face and in real time, right?

Jorjeta Jetcheva: Yeah, yeah, absolutely. And you know, I have to keep reiterating to them. You know, you're doing, let's say someone's doing a thesis, you're some amazing work, but you know my first two thesis students, so they published their work in 2022, and things have moved so far ahead since then. So you have to understand that, you know your work may be obsolete much faster than, you. Any time in the past. Fortunately, they have great jobs and they have advanced much beyond that themselves. And no matter what work we do, I tell them what you'll learn and your ability to think through problems and your just intentional learning and seeing yourself as a lifelong learner is what you're gonna get away from all of these experiences.

Vincent del Casino: Yeah, I can't believe anybody in the fields you're related to. I think all of us are lifelong learners, but I think the intensity of change in the field, I imagine for students, it can be overwhelming from time to time as well, both in excitement to work on these things, but also where is artificial intelligence going next, and what does this mean for me, and what are the guardrails we need to bring to the table? How did those conversations get into your classroom as well? Because I imagine there are ethical questions that come up over time as you do this work also.

Jorjeta Jetcheva: Yeah, absolutely. And every kind of module in the class has a ethics and bias discussion. Because certainly, students learning about the technical aspects of natural language processing and machine learning are going to be on the ground floor with developments in industry, or research, or academia. And we need people to be thinking about this at every step. In the process, because biases can sneak in at every step in the processes. For example, we had a project earlier this semester on experimenting with different ways to convert words to numerical vectors. And that carries some kind of meaning, because in order to run machine learning algorithms, you need everything to be in a numerical format. So we convert words two vectors. And some of these algorithms. Of course, people are trying to fix some of these issues, but if you just compare the meanings of the words, you see that certain names are more closely associated with the word bad, for example, than good. So even if you are someone, a machine learning engineer who is working on a project, and your part is I'm just creating numerical vectors for words, they all make sense, you know, or the ones that you need make sense, so is the word for, you know… you know doctor and medical, are these related in a mathematical way? But if you didn't look at other aspects of relatedness, that then usher bias into later stages in the process, because there are later algorithms that are just maybe doing some kind of analysis of positivity of the text, right? At some point. You know, and if your resume, along with your name and other, you know, demographic aspects or aspects from your experience somehow gets a more negative score because these words are more closely, you now, are kind of in the negative spectrum of the, you know, vector space, so to speak, you will be affected.

Vincent del Casino: So that's really interesting, because when we talk about natural language processing, there is an assumption that we're working with language. And we are working with languages, but we're also working with math simultaneously. And so there's a translation that goes on in that process that then has to be translated back out again, right? So it's like you take language... You create these vectors, right? And then they're gonna produce language again on the other side. So that's like a incredibly complicated process. And I think many people miss that in part because the word natural makes it feel like you're engaging directly with the language, but there's this translation piece. So that like, that's tremendously important. You know, how- how does that start to play into then, like the training of students to get them to see that, but also to your point, ask real questions about what that translation does to the language on the other side.

Jorjeta Jetcheva: Yeah, and you know, students ask really great questions, many of which we don't have answers for. So, you know a lot of times we're feeding text into a multi-stage mathematical process, basically, and there's just so many mathematical operations that are overlapping in ways that we cannot, they're not intuitive for humans. And certainly in the early stages and even now We're almost surprised that. This mathematical jumble ends up in something that has meaning on the other side. So they ask questions, well, we've put these matrices here to kind of capture knowledge from the input text and then do something, but how do we know they actually do that? And I have to tell them, well there's some intuition about how we structure these models. But whether this is how they're actually operating, we don't know. And there's a lot of ongoing work trying to understand what is happening inside. And a lot that work, I mean, inside there are a lot numbers, high dimensional numbers, just matrices with hundreds of dimensions. And then something gets extracted and people try to visualize it and somehow convert it to something that may give us some sense of what is happening inside the network. And in some cases, there's some kind of narrow ways in which we can see something meaningful. You know, overall, we still don't, you know, don't have, you know, ability to track how and why this works mathematically as well as it does.

Vincent del Casino: Which is really interesting because so much now is in literal translation between languages also. So what's happening now is the ability of translation machines to move from English to another language and back again. I've studied multiple language, you know, the space of idiom and colloquialisms, very, very difficult to get a hold of. I imagine you understand this well. And we have, our students come from a wide range of backgrounds. And so. They understand these languages. And yet, we're seeing, as far as we can tell, again, in the public eye, more quote unquote accuracy as things move from one language to another. But what an incredibly difficult thing to do, right? Especially when it's raining cats and dogs. How do you easily translate that? Because that's a multi-layered problem. There's a literal word problem. Then there's a meaning problem there. Are we at a place where the matrices can do that work also?

Jorjeta Jetcheva: To some extent, but not nearly as well, of course, as humans do, again, you know, have the model seen enough examples of this kind of thing to be able to really capture it, and definitely models get distracted by words. So this is one thing that has been observed. So there's some words that cause a distraction, and then the model cannot perform the main function that it's trying to do. So yeah, language is very complicated. I had a student who proposed a thesis project where she was working on low-resource languages, so languages which have not been, for which large language models don't work well because they haven't been trained very much. And she did this for Gujarati, for the Gujarat language, and it was great. She collected, actually, Gujarati school place from some of the local high schools in her hometown to do part of that training and experimentation. Ultimately, that culminated an experiment where there was a small HR data set to try to see, well, how well does, even with additional training with additional data, how well do these models perform? And then she gave it to a number of native speakers to kind of evaluate how well is this even. I mean this HR, HR is kind of a... Pretty straightforward topic, there are no fancy idioms, and many of the words were very awkward, and the meaning, one of the meanings of the word may have fit into the sentence, but still the conversation was very awkward. So there's so many, especially in some languages, many issues with that.

Vincent del Casino: So where is your work going next? What are the things that are top of mind for you? What are you and your students working on, you know, for you, like what's really interesting next question?

Jorjeta Jetcheva: So, you know, a large part of my interest is student success, and I actually realized I didn't fully answer your question about the NSF SSTEM program. So we had 20 rising high school seniors on our campus last summer, and we taught them Python, and we had them build virtual assistants based on open source large language models. So they needed to do some coding. And they proposed their own applications and discovered multiple ways in which the large language models hallucinate. I mean, it was extremely impressive. So, you know, one of the projects, these were small group projects, was about show me nearby murals. And the model would just invent things. So, they're coming to San José State, so some of them, and then will recruit others in majoring in computer engineering, software engineering, computer science. And we hope to use that as a way to kind of see how can we use AI from the early stages and help them to start doing AI research early in their college career. So one of my interests is how do we do research very early on with undergraduates and with larger groups of them. And because they have amazing ideas, I want to see how to take their ideas. And then translate them into research projects. Because again, they source from their unique lived experiences. They come up with things that none of us would, and are just so much more interesting. But one of my concerns in general that I would like at least some part of my research effort to go towards is bias. So if we are, you know barreling towards the future where our world is going to be automated and decision-making is going to be heavily influenced or automated with AI. We need to make sure that these decisions don't disadvantage parts of the population. So this is something that I want to figure out how to better address through my research because at some point it will be too late. You know once everything is already out to me. Well, it's kind of working it becomes harder to change it. I think we need to, you know, from the ground up try to affect it.

Vincent del Casino: Well, it's really interesting and I imagine these questions are really important to the students. And I love the concept of hallucination. I just find it fascinating that we kind of, again, use these words sometimes to talk about these things. But like when they're out there pulling from so many different things and trying to make relationships within the machines. But I imagine the students are like, they see that and probably go, wow, that's going to really impact people's lives. And that's what I love about our San José State students. And to your point. When they bring different ideas from their own lived experience into the room, they can also connect more deeply to the learning they're involved in as well, I imagine.

Jorjeta Jetcheva: Yeah, yeah, exactly. So we, you know, we want to serve as that bridge between them and technology where they can truly contribute their creativity because we need that in the world. Currently, technology is being developed by, you, know, a pretty, you now, male and white dominated teams and it really, we really miss out on the just wide range of opportunities that we have to solve so many problems. Because your lived experiences don't just affect how you solve problems. They affect what problems you choose. It's really the impact is going to be huge that our students can make.

Vincent del Casino: Awesome. Well, I thank you so much for your time. I really appreciate this really engaging conversation.

Jorjeta Jetcheva: Thank you so much for having me. It's been a pleasure.