0:00All right, hey everybody, how are you doing? We're super, super pumped to have you here to talk about JEV in the database here at MotherDuck. Just as a reminder about what we're gonna talk about, I'm gonna just throw this up here. We have a awesome, amazing model called JEV,
0:21which is a new type one LLM type model coming from our friends at typesafe. ai. We have implemented it into MotherDuck. It is both the fastest and the cheapest and the most accurate at the task we've run it against so far, which is really, really exciting. Usually when we get new LLMs, we see that they are making trade-offs
0:45on one of those three at least. And in this case, we get no trade-offs and kind of a new way of thinking. I really love that now we get to kind of think about it more as like a component that we're building into workflows instead of a kind of black box that we put prompts into. Really, really exciting. With me here, we have Hamilton on our engineering side. Hamilton, say hello to our audience. Hello, everybody.
1:10Thank you, Hamilton. He'll talk more about it later, as well as Dumpke on our DevRel team. Dumpke, say hi. Hey, how's it going, folks? Good to be here. Yo, all right. Thanks for joining us late in your day, Dumpke. It's amazing to get the team here across multiple time zones. Those of you in Europe, thank you for joining us at the end of your day. And for those of you in the US, we're super pumped to be here
1:33right in the middle or the start of it. All right. So I'm gonna quickly just jump into what MotherDuck is. We'll keep this really brief for those of you that don't know. So MotherDuck is building a database in the cloud based on DuckDB. We like to think about this as infrastructure for answers. And I'm just gonna look at a few slides here, and then I'm gonna throw it over to Hamilton
1:57so we can do really, really cool, fun stuff. The first thing that we've been observing is that agents and humans don't write queries the same way. When you get your agent running, it's gonna run a bunch of queries to try to understand the data and give itself context. That is very different from a human that already has lots of kind of implicit context. And so they just think about and use the database differently. That's one point. Another point is we have DuckDB.
2:22DuckDB is this amazing ultra-fast single-node query engine. You can kind of run it anywhere. It vertically scales really nicely. And one thing that's different about it than maybe other OLTP databases that you're familiar with is that it saturates immediately on a single query because it vectorizes everything and runs it across the entire CPU, which is really cool. It means it's really fast. What MotherDuck is building on top of this, you know, the brilliant part
2:46is we really think about three parts. Serverless compute, right, which is awesome and means you don't have to think about it. You can get as many nodes as you want, as quickly as you want. Hypertenant architecture so that your loads are isolated. This is great for agents. We love our agents being isolated. We don't love writing a query and then blocking everybody else.
3:08And, you know, if you all are anything like me, I am running lots of agents running against my MotherDuck database, and I don't really look at what they're doing unless there's a problem. And so having this nice little sandbox is very powerful. And the last thing is agent-native tools. You know, with MCP server, we have a great CLI that we just launched a few weeks ago as an extension of kind of like the DuckDB CLI.
3:32And then, you know, dives and flights, which are other ways for you to interact inside of, with AI inside our database. All right, so I'm gonna stop there in terms of my sharing. And I'm gonna throw it to Hamilton, who's gonna tell us a little bit about where things started here and, you know, what he's thinking about and what's super exciting.
3:56Awesome, okay, thanks, Jacob. Of course. Hi, everybody. So I'm gonna talk a little bit about Jev and this new SQL function we have called PromptJev. We just released it last Monday. So that was eight days ago from this stream. A week before that, I think it was on a Wednesday, I got access to this new Neo lab called TypeSafe AI and their model, Jev.
4:22And the way that you can think about it is Jev kind of, it's essentially, it's a large language model, like the ones that you actually use, but instead of, you know, returning a well-reasoned string of texts, it returns basically typed decisions. And what that means is you can't treat it quite the way that you would treat like Claude or ChatGPT, but because of its architecture
4:46and because of its like extremely constrained output, they're able to make it extremely fast and extremely cheap, but the same kind of general intelligence that you would get from an LLM. And I think that's really exciting. So when I discovered this, when I heard about it, I thought, whoa, okay, this is really interesting. I've been in the analytics space for a really, really long time.
5:09When I first joined the industry, the data scientists were not yet a thing. My first job, I was a statistician. That was my background. And it has always been difficult, I think, to find the sweet spot of pulling structure out of unstructured data. You know, it's become much easier to collect data in our companies. And we are told that it's important for us to collect as much data as we can
5:34to help make good product decisions. But there's this huge gap between the ease of collecting the data and the ease of actually doing something useful. The one area where it's most challenging has actually been pulling structure out of unstructured text data. There's a million cases of this, right? Product descriptions, comments from social media, or sales calls, customer tickets, anything where humans are actually outputting stuff,
5:59and increasingly agents. It's quite hard to pull structure out. And so a lot of companies just don't do anything with this data. It sort of sits there in their data warehouse without anything good happening to it. The status quo before Jev was that if you were somebody that understood machine learning, you could basically fine-tune a classifier
6:22to put a label next to that data. To do that, you would have had to have hand-labeled your data set first, your training set, and then use that to basically fine-tune. That requires expertise and a lot of time and maintenance costs that most people just don't wanna put in. On the other end, with modern large language models, we thought, wow, we could just prompt for each row and get a label out.
6:46That is such nice ergonomics, right? Because you don't have to think about it too deeply. You could just sort of basically just talk to your database and good things happen. But large language models are incredible, but they're kind of, the ones that are on the market are pretty slow for this kind of task at scale. If you've got a million, 10 million, 100 million rows, you're basically asking someone to sit and think for about each one and then find some way
7:10to pull out the structure from it. So it ends up being too costly and too slow. So most people don't do that as well. So JEV, what I'm excited about with JEV is it gives you all the ergonomics of the large language model. You can prompt it in natural language and treat it like a precocious intern that kind of knows about the world, but it has all the speed and joy of a traditional machine learning classifier.
7:33That makes it substantially faster to use for this kind of pulling out structure task, and also way, way cheaper as well. So you don't really have to sweat whether or not you're pouring a ton of money into just pulling, you know, making, creating labels on data. So that's why I'm excited. That's why I was excited about JEV. I thought, wow, if this is actually as efficient as they say
7:55we can directly integrate it into SQL and you could just run queries in prompt and you'll barely feel it basically. So that was the premise. And so about an hour after I got access to JEV, I put together a DuckDB extension because, well, the ducks at DuckDB shop, the greatest, greatest analytical database ever made. I'm not biased at all, obviously, but it's a real joy to use.
8:19And DuckDB is already incredibly fast. So I thought, let's pair fast LLM with fast database and see what happens. And, you know, I'm terminally online. So I was on X and posted it there and it kind of blew up. And I realized, oh, there, I think there is like a huge unmet latent need here that I think we can probably give customers. So five days later, we actually released support
8:41for prompt JEV, which is the table function and I'm, or excuse me, the scalar function that directly integrates with MotherDuck SQL. I'm gonna pass it to Dunkey to just kind of show you how it works, how it feels and like what you can get out of it. All right, thanks. Yeah, let's get started with seeing some JEV in action. So I think, first of all, what's very nice is kind of the ergonomics, right?
9:06Of having JEV in accessible in SQL in MotherDuck. And let me see, I think Jacob, you probably need to hit share on my screen. There we go, there we go. Thank you for the verbal prompt, you're good. Yeah, no worries, no worries. I'm used to prompting these days, so.
9:30Basically what I kind of want to show you is we have a large table with job descriptions, right? And like Hamilton said, job descriptions, like support tickets and all these kinds of like large text fields are just perfect for interacting with someone like JEV to classify them or extract information from them. And so let's say we wanna see which of these job descriptions
9:56are based in London. Now, normally what you could do in SQL is you could have something like, like, or I like, um, London and, you know, we can get a couple of these jobs in, let's see if my duckling has started up here. Yep, there we go. We see that the, you know, we got a ton of jobs in London
10:20and we can do something similar with JEV. Obviously the difference is gonna be that we're not looking at actual text. We're looking at something that's more of a, a query, right? So it's not, it has a semantic understanding of what is going on here. And we're running this over 10,000 rows here,
10:45took about 15 seconds. So yes, it's slower than, you know, just doing an I like, but it has an actual semantic understanding of what is going on. Now for something like just London, that's maybe not as relevant, right? But we could also say something like,
11:07is this a remote job with a base in London? And as you saw before, like we get a score out of this, right, it's not a yes, no, it's not a binary thing. It is a confidence that the model has to find this. And maybe should have toned this down a little bit
11:31because now we're writing 15 seconds every time, but you can see that this is very likely to be a remote job. Let's see, this role is fully remote, but headquartered in London. So exactly what we're looking for, something that you would never be able to find with just some text extraction.
11:53It took us 15 seconds to do this for 10,000 jobs. So that's pretty cool as far as I'm concerned. Now let's look at some other things where it gets even harder with like standard text extraction, right? The famous example, obviously, maybe after, I don't know, Rust, the worst chosen name for a language to find
12:17on whether it's Google or any kind of search engine is Go because it's everywhere. And so obviously, looking at Go, we'll get stuff like governance, Go, all these kinds of things that we're not really looking for. Yeah, sure, we can add in like a space in there, but then we miss the stuff that has a dot after a period.
12:41And so instead what we can do with something like Jev is we can look at this Go thing and see all the different concepts that are in there that we similar conflate when we search for terms like Go. We can say like, is this a Go programming language? Is this in a go-to-market team? Is this a governance role? And what we've done here, and this requires a little bit of fine tuning
13:06on your side probably, we can say we're taking a confidence score because it brings back that confidence score. And this is the nice thing about the MotherDuck implementation of this. It's just a score, basically, a double. And so we can put a, we can turn this into a binary setting if we're looking for like a range
13:31to determine if this is actually something we wanna take into our query or not. And so what you'll see is we'll get these things out and here we get a true or false back. So not just a confidence score, we set a threshold basically to find what we're looking for. And so this supposedly is a go-to-market role.
13:56Obviously, I still don't fully understand what go-to-market means. So it could be that it's, we can't really find this in here, but obviously a Golang should be much easier for us to understand. Let's see, not a lot of those. This is just data jobs as well, right? So that's probably why. Maybe let's go for a governance role instead here.
14:20This is a government's role. A privacy office. Now that sounds definitely like a governance role. So it gets these kinds of things as well, because again, we're looking at semantics, not just text matching, basically.
14:45And now when we take that a little bit further, we can, let's look at the way we do this, right? At MotherDuck. So we have this from Jev function. It takes a, in essence, a description, or sorry, it takes a text field and then a question about that text field. Now, if we go a little bit further,
15:08we can add in some additional features to make this more interesting, right? So instead of doing that- Do you want to talk about the types? Can you just like talk about the types that are available to us? Yeah, exactly. I was just going to do that. Perfect. So basically what we've done is, this is a little bit of a different way of asking this. And so we can add in a lot of these features
15:32that we want to measure and we'll measure it over the same text description. So again, the way they've implemented this, this saves a ton of going back and forth because it just caches this input basically when it's asking questions about it. And so then we can ask questions of different types. And so this is the most default one, the most normal one, a no,
15:56which gives you basically a, it's basically a yes, no question, right? And it gives back a confidence score, how likely it is to be a yes or no. And the other types that we have is, for example, a choice type, which will, where you give it a set of parameters
16:17up to, I think, 256 or 255, if I'm correct. And so you could give it, for example, a ton of categories. Let's say we're talking about job descriptions. We could give it a number of like programming languages or all kinds of tags or categories that we're interested in. And we use that as a choice.
16:40And then the final one I am blanking on right now. We had the null, the choice, and help me out there, Jacob, you know? I'm picking it, hang on. You know what? I'm gonna go reference the blog we just wrote today, which has all three of them.
17:02Let's see, we've got a choice, null, and score. Oh, score, yes, of course, yeah, yeah. So that basically is another kind of like categorical scoring of a field, right? So let's say you're looking at support tickets. Is the urgency of that support ticket, is it low, medium, or high?
17:27And then again, it can give a confidence about that score. It's a re-ranker, basically. Yeah, exactly, yeah. And so one of the things that I think is very nice is that because it's so cheap, it allows you to think differently about how you query, right? And so let's say we have these job descriptions, right?
17:51And so I'm taking a little bit less jobs because we're gonna do something different here. So for the Jev part, it doesn't really matter how much we're taking, but you can see that we can use our generic prompt function to do some more compute-intensive prompting. We can use a larger model, I've taken nano here,
18:16but maybe if you have a more difficult task, you can take an even more expensive model. And so basically what we're doing is that we're saying in the first step, do we actually think that there's a salary here to extract? Because if it's not there, then why bother with the more expensive model to even do this?
18:37And so then we go from that baseline of filtering to actually enriching it and extracting the actual value. So that's the one thing that Jev can't do, right? Despite all the things you see on Twitter with like Jev wrote my PhD thesis or whatever people are saying, it just gives you a score. It doesn't give you actual text.
19:02So what we're doing instead is we're taking this enriched part. So we're first, let's see what we get with just the Jev one. Super fast again, under a hundred seconds. And you see that these are probably, let's see, where are we with the description?
19:23So this should have something like a salary in here. Yep, exactly. So there's a range here. And so obviously with salaries, it gets a little bit more complex because usually you have like ranges, you have like different currencies, all these kinds of things. And so that's what we wanna do with this enriched part. We're taking a more expensive model.
19:48But again, like we discussed, right? So just to show you the difference, right? We can have a salary period. So monthly, weekly, or hourly, for example. with our more expensive model, but especially these kinds of categories are perfect for JEV as well. So if that's the only thing you'd want to extract and like tie it to one of these groups, then this,
20:12the choice one is a perfect way to go and you'd be much faster actually. And so now instead of processing the full 100 rows, we were only processing 33 rows. So that definitely saves a lot of money on using these more expensive models. And so what you'll get
20:31back here is basically a strut, right, or we see that this is Australian dollars apparently. And they both have the same salary period, right? So they take the JEV and GPT nano, five nano took the same, extracted the same value here.
20:56Now, one thing that we talked about a little bit, the final thing that I think is very interesting to show is that you can use this as a re-ranker as well, right? So I added to this data set, I used our embedding function. So there is a, or is it descriptions embedded column.
21:17And the nice thing here is that we can first do a ranking, right? Let me show you what that looks like. And basically this uses an embedder model on the description. And what do we have as a query?
21:36So we wanted to see data positions in Amsterdam using Golang. So it looks for these, these terms and looks at the similarity basically. And so apparently this is the most similar job.
21:54So I'm assuming it has Amsterdam here somewhere, Golang. It's a little bit of a wall of text, so I can't really see this now, but that's kind of how the embedder works, right? Like we calculate a distance between the query text and the actual description text based on this giant matrix of
22:16values. Now, what we can do with Jev is once we have our set of things where we're like, hey, this is what is actually, so let me explain it a little bit differently, right? Because the thing with embedders is that they will rank anything for you. So they will come up
22:40with a distance to something and they will come up with a score, even if it's not relevant at all. So what you can do is take this ordered list basically of things that might not be relevant, even though they have a distance between the query and the actual embedding text, embedded text,
23:01and then use something like Jev to re-rank that. So that allows you, if there is something else going on, let's first see how we get our re-ranking here. What did I do? I added two froms. That doesn't work.
23:24It allows you to use a model like Jev to just take out all the results that are not relevant and so we could do something like that as well. So these are the results re-ranked
23:41and then basically let's say if we have, I want to see positions on pizza, whoops, that's my cat's block on pizza chef on Mars. Let's see if we
24:04get anything for that. And now it's still, you can see that it ranked things and it will still find things with pizza probably. So we do have to maybe fine tune our question for Jev a little bit. But the thing is that we can fine tune our actual score and say,
24:26like, hey, is this actually relevant to the query? And if it's not, of course, there's data jobs for anything, right? So there's likely to be stuff on Mars, there's likely to be stuff with actual pizzas. But you can use this-
24:45Oh, yeah. Martian ERP. Okay. There it is. Yeah. And it was something about burgers, I guess. So something with food and Mars University. You always find that one thing. All right. That's not too bad. I think this is a very interesting pattern. Just to go back to it,
25:08you can super fast, you can find the job descriptions with these embeddings. And so you can calculate the distance, but then you can ask the actual question of, should I even show this to my end user? And is this actually relevant? And that allows you to go from
25:29generic distance rank things to saying, hey, actually, there's nothing relevant for you to see here. So, yeah, I hope that this has given you some insights into how to use Jev with MotherDuck. I think for any text classification, for re-ranking of search queries, those kinds of
25:50things, for choices, for making a staggering of when to use a more expensive model, I think it's a perfect tool to have in your toolbox. Yeah. Amazing. I mean, I think it's so cool to be able to. . . A lot of things that we think about
26:11is like hybrid search on text, like using full-text search in combination with embeddings. And now we get another kind of modality of that, which is re-ranking it with Jev, which is awesome. I think. . . Let's do some questions here. I think we've got some good ones. Dunkey, are you going to write a blog or is this going to land in a cookbook? Where's all those
26:35really cool examples? Yeah, I just realized I should. . . I made a big part of this today, so I realized like, hey, this is good material for a blog, so I'll put this out in a blog. All right. I guess you're now on the list. All right. Jacob, can I actually intervene and just talk a little bit about what we've seen since we've released it? So it's Tuesday today. And so last Monday,
26:58eight days ago, we actually released Jev Support in MotherDuck. And we've been kind of blown away, actually. And I just want to talk a little bit about how our actual customers are using this today in actual production pipelines, because I think it's really interesting. So essentially, we are like many, many, many, many billions of tokens into this integration already.
27:23And we have actual pipeline. . . Customers are running actual pipelines with PromptJev right now, in many of the ways that Dunkey just described. So I think we've seen three high level kind of use cases for. . . Basically, three things that I think really stand out. The top use cases really are about tagging kind of product event data of varying shapes, labeling, and scoring support tickets. We've seen a lot of
27:48customers actually use it for sales calls as well. So I think that is far and away the most common thing. If somebody has text data and they want to do something interesting with it. The second category that I actually was really surprised by is agent productivity. So if you think about agent traces, they're essentially just free form text as well. We have a lot of customers that both have agent products where they have traces where they want to essentially use
28:13them, understand how their agents are working. They also want to check on their own developer productivity and categorize how their developers are using their agents and tokens. That was really interesting. Generally speaking, we have customers of all sizes and stages using it. We have really big established accounts. We've had some interesting cases where these small kind of
28:35dormant ones pop out of nowhere and start talking to us after they refuse to talk to us at all. We have basically everything in between those two poles in the spectrum. We also have a ton of new signups who have actually just jumped right into MotherDuck just to start cranking, throwing CSVs in and actually figuring out with it. So it's very cool. And it's neat to see actual use cases kind of match the mental model that
28:58we had in mind for this already. Totally. Yeah, it's been really cool to see. Thank you, Hamilton, for grounding us a little bit there. Let me throw in, I'm just going to throw in a couple of questions that people have had here. First one is kind of about nuts and bolts. Does any data leave MotherDuck for Jev to work? Is Jev hosted on MotherDuck? Hamilton, I can answer it or you can answer it. I don't really care.
29:23Yeah, I think I saw another question about like, yeah, basically, how does this work? Which is a very valid question. So the type safe people actually did a really smart thing. When they released Jev, they actually released their own hosting for it as well. And so today, Jev is run on their servers. We have a zero data retention policy. And generally, I don't know or think that
29:47they're actually training on anyone else's data. So they only keep the data for the duration of whatever is needed to generate in return the labels, and then they throw it out. That said, Jev is in our prompt Jev function is in preview right now. Because type safe and Jev are such new things. I think they're still working on a lot of the details of actually supporting this at
30:09production scale. And that includes things like HIPAA compliance and things like that. So just be very aware of what your legal and compliance needs actually are before you jump in and using and use Jev in MotherDuck. Yeah, I think that maybe maybe too. Yeah, maybe too. Yeah, maybe to fill in from from the developer perspective, there is it's also only available
30:32in US East one region and US West two. So if you're in Europe, for example, right now, or in Asia Pacific, it's a little bit harder to open, but we are working on support there. Maybe just said, we actually support everywhere, but Europe, basically. And that's primarily
30:52because I also didn't realize, but I think we have we have customers all around. So and we are rolling out EU support on October 6, which I think is next week. And the only thing that's really stopping us right now is just making sure that we're in compliance with GDPR. So just kind of
31:13we're checking the right boxes. That's right. That's right. There's a certain Yeah, you can't just Yeah, yeah, yeah. For these things, we're making sure all the yeah, boxes, check there's two pieces, there's compliance, and there's also latency as well. And so we're also working with the type safe people to figure out when exactly they're planning on supporting other regions, because Jev is really fast. And that's, that's nice. It's not the nicest thing about it. But
31:36it is really nice. And we want to make sure it's fast for everyone. Yeah, totally agree. Good follow up question on this one here. Is it possible to fine tune Jev with my own data on MotherDuck to be more precise? I will start, I'll start this, which is to say, I don't think that fine tuning is the first use case for Jev, I suspect it will be at some point in the future, be able to be able
31:59to do this. That being said, you can put whatever text you want in to the box. And so a lot of the things to think about is like, Dunkey showed us re ranking and embeddings and things like that. Those are all parts you can do to kind of give better context to Jev, right? I don't know if
32:20anything else comes to mind for you, Hamilton. I believe Jev supports 32,000 token context window. So in theory, you can stuff a bit of data in. And I bet you that will tolerate it pretty well. I haven't tried it yet myself. But that might be the equivalent of fine tuning. Yeah, that's I think we're a little bit back to back to prompt engineering, right?
32:41It's a little bit more important to put the right context in your prompt. Yeah, totally. Totally. Okay, question here from Kim. Will prompt Jev work on non-text data? For example, if I had monthly data on a specific portfolio metric, can Jev tell me what the overall trend is, where spikes occurred, why, if it had data fields to query and use an explanation?
33:07I can start. So I think the answer to this is, this goes back to the prompt engineering question, right? So you need to structure it a little bit. What I've actually also seen, actually, I'm not going to bring that up. It's very complicated. So yes, you can structure it inside of a prompt, like, hey, I'm, you know, I have these sets of observations, you know, are these unusual? You can then also give it a question like,
33:32you know, given data set X, you know, what classification is most likely, right? You can give it a set of different categories to score on. Like, is this an outlier? Could be one of them. You know, is this business as usual? Is this something else, right? In terms of why, I think the pattern that Dunkey showed for saying, hey, if confidence is below some threshold,
33:53or you could basically like say, like, you know, do we need to explain this outlier and get a prompt on it, right? And if the answer, if you get above a certain context, let's say 90, right, 90% confidence, then you could pass that to another model immediately and say, okay, now, let's pass a richer context to another model for saying, hey, like, look into the outliers, right? Because what you don't want to do is run this across a huge data set of all potential
34:16possibilities, right? You know, thousands and thousands of rows, you want to give it a very targeted set, right? And so that goes into like the re ranking that was shown earlier as well, like, you know, which ones of these are most likely outliers, but therefore we need to do more analysis on so that when I get the report in my inbox, it's much more tight by the time I'm actually seeing it. I think that that's how I would think about that. It kind of sounds like you're saying we're at the statistician in me is wondering if we're doing like residual calculations
34:40in free text. What a world. Turn that part of your brain off, Hamilton. I think definitely. So the thing I would expect as a next model is, is one that's maybe multimode or at least image based, right? So I think that's maybe the question here, like, is there a trend
34:59in an image? What to do, confirm what Jacob's saying. I think if you put in a table of data, it's likely that it'll be able to answer questions. Although the Jev folks or Typesafe folks do say that that's just not the best use for Jev. But it obviously would make a lot of sense to have
35:20a fast model like this to be able to tag or categorize images. That is definitely one of the next steps, I would say. Yeah, totally agree. It does sound a little bit like a system two type question, like you'd want a reasoning model to. Yeah, I think the thing that I see in that question that I would just highlight with the Jev piece is there's certain questions that
35:41are in our list of things we want to know about that are system one questions. We should ask those first. So then we give a much tighter prompt into our system two models, right? So that way we can actually have that thinking occurring. And maybe to explain a little bit to folks like the
35:59system one, system two thinking, right? It's based on that originally, I guess, a paper from Daniel Kahneman and Tversky, but the book basically thinking fast and slow. And the analogy really that I like for thinking about this is that of a camera, right? If you have a digital camera,
36:19you have an automatic setting and you have a manual setting to really fine tune everything to your needs. And most of the time, the automatic setting allows you to go really fast and take good pictures. But if you have a different type of question, if you have a different specific situation that you want to tune it to, then you need a system two model, basically.
36:41Yep, absolutely. All right, another question here. Can we use PromptJev with a flight to be able to schedule it easily? Tumkey, you want to take this one? Yeah, yeah, sure. I mean, the simple answer is yes, go for it. Amazing. So I think this is a great idea, right? So I would make it part of a
37:05pipeline and it depends a little bit on the type of thing you want to do. But I think we've shown you a couple of patterns to kind of go like, hey, maybe we want to filter some stuff that we're going to use in a separate part of the pipeline with a more expensive model, or we're just going
37:23to classify all our sales calls as they come in every minute and keep the flight running 24-7. That all just works. Yeah. Because it's all just like, it's a SQL function in MotherDuck. Yeah, absolutely. I would also say we have customers already doing this too, just as a proof point. It's a really common way for people to use PromptJev. Yeah, and just for those of you who maybe are less
37:45familiar with the MotherDuck insider terminology, a flight is a pipeline running on a cron inside of MotherDuck. Okay, what else we got here? I'm just scrolling down. Let's see. There's another question here from John about sending sensitive data. What are our thoughts? Hopefully, John, your question was answered. If it's not, drop us a follow-up
38:06and we'll take it there. Will the recording be available? Yes. Okay, let's see. A question here. Any approach on using Jev to make a product similarity score based on images? I think we talked about this. That being said, the trick is actually what Dunphy showed earlier, which is you can pass, you can create those, you can turn those images into embeddings and then use
38:31the embeddings to do that. That is not using Jev, obviously. That'd be using kind of classical embedding, but that is totally possible. I guess it depends on how many images you have. I suspect, though, that next generation of whatever Jev looks like will contain some sort
38:48of multi-modal. Yeah. Let's see here. Yeah, and again, maybe to go back to the way of thinking about this, right? So I think thinking of it in terms of similarity and so thinking of it in terms of distance is probably not the right approach for this model. It's much more
39:07categorical and classification-based. So again, let's say it was available for images, which it's not currently. The better approach would be to what objects are, or is there a car in this image, basically? Or which four of these objects are in this image? I think I did see somebody on
39:31Twitter, correct me if I'm wrong, actually come up with a bunch of labels for text and then use that to get confidence scores and then use similarity to compare it. I'm curious to know whether or not that actually works well in practice, but it's sort of like a human-readable form of embedding where you can actually say why two things are similar. It's kind of an interesting concept. Yeah, I saw a really cool one also on Twitter, I think from like the RoboML
39:55guys. I can't remember. I'll see if I can find it in a second. but they basically, instead of scoring each frame, like in a sports analytics, like soccer or basketball, they were using like a really high end model for scoring one frame every 30 seconds and then scoring the interim frames with Jev. And that was a wild approach for kind of getting,
40:18you know, that way they're able to do it almost more in real time, like not exactly in real time, but almost in real time for image classification, but I kind of doing a hybrid there and exactly how they were doing that. I'm not sure. I think there might've been like building embeddings on different sections of the screen and then like using Jev to re-rank, you know, based on what the ones that were determined by the higher, higher quality model. But there's lots of things
40:41you can start like kind of combining things onto by saying, Oh, if I have, if I, if I, if I can reduce this down into a set with a more expensive model, I can then use Jev to classify it. Right. Which is pretty cool. Here's a good question. Maybe, maybe one thing to point out, like I I'm just seeing this comment from from Ian, which I think is very interesting. He says
41:05I've seen products categorized emoji search. And so you can search for something like green checkbox. Right. And so this is the thing about Jev. So we talked about, it's not for images, but it is for all of Unicode texts. So emojis are Unicode texts. And so that means that they do have a semantic meaning in that model. Right. And so this could be, you know, a very interesting
41:28use case where maybe you want to classify an image with emojis first, and then you can do your similarity ranking. But yeah, just to this point here I think this is a, this is a very interesting use case because, you know, we all know how it's, how it is to search for emojis. You never find the one that you're, you're looking for. And it's nice to see the speed of Jev being
41:53used in this way to search for someone. And I did see linear did deploy that as a feature. Yeah, yeah, yeah. No, you're in linear. If you're in linear and you're looking for the perfect emoji to describe the status of your, the work you're doing, you can now use Jev as part of that. Good one. That's great. All right. This good question for John, how do you handle version
42:14changes? Like if the model or function changes, do old labels get rerun? How do you save the model version of each label? I see this as like a classical data engineering question. Don't you want to take this one? Yeah, I think so this, this is the case, for example, for the embeddings that we showed, right? If you, if you have something like an embedding function, you need to make sure that
42:38the embedding that you run at runtime is the same model that you used to like classify or used on all your existing corpus, basically. And so it doesn't necessarily change anything. The way we
42:57handle that now with our existing prompt function, for example, for the more expensive models is that you specify a model. And I think depending on your use case, yes, you might want to rerun something if it's really more accurate, but I would say start with making your own evals first,
43:22maybe classify with a more expensive models for an X percentage of your, your data set, and then use Jev to kind of see how accurate that, that is compared to the expensive model. Yeah. And if you do, by the way, in the, in the launch post that we shipped for Jev, which I'm going to share a link for in a second at the very bottom, there's a, there's a script if you want to run to compare, you know, across different models.
43:48Here we go. Link sent. Okay. Really good question though. Let me pull the one away. Let's see here. Let's see here. Anything else? Oh yeah. We talked about this, but we'll just talk about it again, because I think it's important. Any plans to make prompt Jev available in the EU regions?
44:13Hamilton, what was that date again? October 6th is when we're planning on rolling out support. Amazing. So there we go. We want, we want everyone to be able to enjoy it just like we have in our customers. I don't see anything like. . .
44:32There's a good question, Jacob, at the end about relying on a BERT model versus Jev, right? I think again, yeah, no, it's a, it's a great question. And again, like with a lot of existing models, Jev has not been trained on any data that you have that is not public data,
44:54right? So if you feel that you have a specific data set that is very unique to your company, yes, please use a BERT model that you have trained or fine-tuned on your, on your data set. I think the thing about Jev is that, like I mentioned before, with the camera analogy, it is super
45:14fast and easy to make a photo basically on the automatic setting, but it's not going to be as good as when you take the time to create a manual setting that is very specific to that situation to take your photo, right? So Jev, the unique thing about Jev is mostly its price and its
45:37speed, I would say, and that just opens up a lot of new use cases. Whereas if you do have specific data that's unique to your company or a tagging or classification pipeline that is very important and, and I don't know, is used on x-rays or whatever, please don't use Jev for that.
45:58Because that's, that's a different use case. Jev is for when you want quick classifications that are mostly pretty to very good. Yeah, I would agree there. Thank you. Thank you, Dunkey, for, for taking on that one. Lots of other good questions I've seen here, but nothing that we're going to pick up today. Thanks everybody for joining us. We're just,
46:22we're just about at time. Hopefully everyone found this awesome and is enjoying it. Again, just a reminder, go, go check out motherduck.com. I think we even have a landing page. Let me just see motherduck.com slash Jev. I think if you happen to be wanting to learn more and kind of
46:40get our perspective on this, I'll share the link here and, and here, and one more. Amazing. Definitely check it out. We're doing cool stuff. We're continuing to do cool stuff. I think, do you want to tease something Hamilton? Like did, did I hear that it's getting faster?
47:01Yes. So in our like preliminary benchmarks, we found that for similar classification tasks, we found that Jev was about 50 times faster and about 1% of the cost. Those numbers can vary quite a bit. But we actually recently saw that we can actually potentially double the, the
47:25efficiency with the same exact quality responses. So I think as of this week, Jev will be twice as fast on top of the already extremely fast. So very excited about that. Yeah. Nice little things, nice little engineering wins. I think we can get in there, which is really, really cool. All right. Thanks everybody. We, we appreciate you joining us. Hamilton, Dunkey, thank you for
47:47joining on short notice and building something awesome, sharing with the people. This will be recorded and shared. Look forward to finding and interacting with you all on Twitter slash X and LinkedIn and, and more. Thanks everybody. Thanks.
