EP 038 Cutting edge Deep Learning w/ Nyckel

Joining us today is Oscar BeijBom, co-founder of Nyckel to discuss all things Machine Learning.

Oscar is the co-founder and ML lead at Nyckel, the lightning-fast API for Machine Learning. He has 20 years of experience shipping ML products for a wide range of products including self-driving cars, airbag bicycle helmets, coral reef monitoring, and microscopy. His academic work on deep learning has been cited over 5000 times and includes topics on object detection, domain transfer learning and fine-grained visual categorization.

Subscribe to the PEAPOD for more exciting episodes like this!

Transcript:

Mike:
Welcome everybody. Thank you so much for joining the show today. I have a really interesting guest. His name is Oscar Beijbom. He is doing a bunch of stuff in machine learning and he is a co-founder of a really cool machine learning company. I’m calling it a machine learning company. Maybe I’ll get corrected during the show, but it’s called Nyckel and it is a no code, it’s like a no-code tool that you can use to allow you access to not only creating your own deep learning or machine learning models, but to use them, to actually invoke them in your code.

Mike:
So it’s just a really nice, it’s a nice way to get up and running quick with machine learning. That’s my take. We’re going to talk to Oscar, see what he actually has to say about my take here. But Oscar, thank you so much and welcome to the show.

Oscar:
Thank you. Thank you for having me.

Mike:
Yeah. So Oscar, could you just tell us a little bit about Nyckel, like whose your base? Who are you trying to target? Just to give you an idea, we’ve got thousands of developers who listen into our show. Most of the people are into embedded or they’re getting into embedded electronics. We range from people who are just getting started into coding Arduino, to people who have been coding for years. Some people are like garage entrepreneurs, building something new. And other people are working for corporations, and other people are just hobbyists.

Mike:
So it’s a big range of people, but more and more, I find that people are interested in cutting-edge technologies, for example. Like machine learning and deep learning. And so, I don’t know if you could tell us about Nyckel from maybe the perspective that would be good for ears like that?

Oscar:
I mean, so Nyckel is our attempt … Let me back up a little bit. So machine learning took a big leap forward in 2012 or so when deep learning really started covering a couple of breakthrough academic papers in deep learning. And that’s been my career, I worked with self-driving cars and tried to build a AI to drive a car for the last six years. But it surprised me as we’re talking to friends that are developers just general software developers, how inaccessible it still is. How like much lingo, even if you go to the tools, there’s a big ecosystem of companies trying to put that machine learning in a box and make it really, really easy.

Oscar:
But even those companies, we found use too much machine learning based lingo that had too many assumptions on knowledge of best practices that you need to know when you build a machine learning model. So Nyckel’s on the extreme end of making it accessible to non-experts, or non-machine learning experts, meaning general software developers basically, or product managers or something like that.

Mike:
Okay. Yeah. No, that makes sense. So I know I’ve heard the term thrown around, like intelligence as a service. Or I don’t know if that’s the catchphrase, but basically it sounds like what you’re able to do with Nyckel is you are able to not only train your own … So you’re not using somebody else’s model, you’re creating your own model using your own data.

Oscar:
Yes.

Mike:
And you are training it on the platform. So it’s not like I need the GPU resources to train this model. I can train it on your platform and then I can also deploy it on your platform. And if I understand correctly, all I have to do is make a web call to your API. And then I’m able to then basically get the result of like, let’s say I train a model to determine different resistor types.

Mike:
I upload a bazillion pictures of resistors and I want to train a model that’s going to tell me, hey, is this a 10 ohm resistor or a 10k resistor? Then I can go in and I can train this model, upload a bunch of data. And then, I can take a picture, send that picture to Nyckel. And then Nyckel’s going to come back with some level of confidence and say, hey, this is a resistor, this isn’t a resistor. That’s the basic core idea here? Is that about right?

Oscar:
Yeah, that’s exactly right. And I think the difference there between other services that we don’t … If you think about Google’s, Google has a vertical Vertex AI, which is superficially it sounds very similar in the sense that it abstracts away everything. But they still have steps we have to go in, you have to click, train. You have to evaluate the model. You have to click deploy.

Oscar:
With Nyckel, after every sample you upload to Nyckel, we retrain and immediately redeploy the model to the InvokeEndpoint. You can literally, you train in about 10 seconds, it’s deployed immediately. And then you can start working on it. And it also allows you to do things like, you train the first model and then you start using it in production, right? And then you say okay, here’s this mistake Nyckel is making. And then you pose those back to Nyckel. We automatically retrain and redeploy for you. So it allows you to close that learning loop quickly.

Mike:
Yeah. I tell you what, I’ve been playing around with deep learning for years and machine learning stuff, just these models. And I was really impressed with Nyckel, because I watched the intro video. I signed up for an account and it makes a claim, hey, start your model in five minutes. I’m like, yeah, right. You know? So I was like, I’ve heard that before. But no, it was true to the call. Man, that one-minute intro video just lays it out. Literally you can learn how to use the platform in a minute and it’s not like I’m a genius. You know what I mean? There it is, you upload your data. You just mark it. Hey, is it this, or is it that? And you roll.

Mike:
The interface was so insanely straightforward. I’m like, okay, I get it. I mean, it took all of a minute. So I was pretty excited about that. I think the things that really struck out to me was one, the interface was super easy. I didn’t have to upload a ton of data, which was nice. And I think obviously, and tell me if I’m wrong, but if I understand correctly, there’s a trade-off there. There’s a lot of people who say, no, you need just tons and tons and tons of data to have any type of useful model. And then I hear other people say, well, it’s not necessarily true.

But I guess what I like is the option to start with the data I have. And then I can always add later. And then like you mentioned, it seems like updating the model isn’t some crazy thing. You’re just telling it no, hey, you made these errors and then that’s going to go into making the model better. So those are the things that … So anybody listening to this right now and they’re thinking to themselves like, oh, machine learning, no way. How am I going to integrate that? It’s like, I’ll tell you what, check it out. It really wasn’t that crazy. I don’t know.

And I think what was the minimum data count? Like a hundred or 500?

Oscar:
No, it’s actually just four.

Mikw:
Four, all right, there you go.

Oscar:
You need two examples per class. So if you just have a binary classification problem, you just need to give us two of each and we’ll train the first model. Now obviously I wouldn’t recommend going to production with that. But we like to train as soon as possible to give people a sense, because as you’re annotating more data, you keep seeing the predictions of Nyckel like of the previous versions of the model. So you can, as you’re annotating, you get an intuitive sense of the types of errors it’s making. And where it’s doing well and where it’s doing poorly.

Oscar:
Obviously we also track accuracy and all these statistics. But by seeing the results as you’re annotating, you get familiar with the model in a nice way, I think. So that’s why we train very early and just keep retraining and retraining and retraining as you go along.

Mike:
That’s really cool. As the model gets larger and you are making a call to the API, what type of turnaround time are you looking at to get a response back? How rapid can these things give an answer? And this is maybe an unfair question, but especially when working with like embedded devices, is that a known time or is that a known window kind of thing or?

Oscar:
Yeah. So that’s a good question. So I mean, first of all, I think to your audience, we currently, as you said, the only way to invoke a Nyckel function is through the API. So we currently don’t have a way to export the model and put it on the device. That’s on our roadmap. And we are in fact looking for beta testers of that or people that are interested in helping us define. Because we want to do it the right way. We want to do it quote, unquote, the Nyckel way where it’s as painless as possible, so we find the right abstractions. We’re not confident we have those yet. And we’re looking for beta testers.

Oscar:
So that’s one thing, to the invoke point. So even if you have an embedded device today, you basically need to have internet connection, be able to call the API.

Mike:
Yeah. And to be honest Oscar, that’s really pretty common is what most people are putting some type of wifi enabled board. In a lot of cases people are using like the ESP8266 or something like that. So it’s really pretty common.

Oscar:
And in those cases, we strive for about 300 millisecond evoke latency on our side. So it depends on what you’re doing, right? If you’re actually controlling the actuators of a robot, that’s probably too slow.

Oscar:
And then you asked about retraining time. So currently it’s going to be very fast. Depending on the price plan you’re on, we support up to 50,000 training samples. And our goal is to retrain in less than 10 minutes, even at that level.

Mike:
Oh wow. That’s impressive.

Oscar:
Yeah.

Mike:
So actually the algorithm behind what you guys are doing, is it like a vanilla model? Or is it like using some type of transfer learning? Like it’s already been pre-trained on like ImageNet or whatever that kind of thing is [crosstalk 00:09:18]?

Oscar:
Right.

Mike:
It’s ready to go type thing, or is it just straight up, hey, I’m learning from what you give me?

Oscar:
No, it’s very much transfer learning. And that’s one of the like, if you think of Nyckel from like a machine learning theory perspective, the reason we think Nyckel or something like Nyckel would be big in the future is because the bigger these pre-train networks are, and you hear about them all the time, right? From the big AI houses, like OpenAI or Google, these big, big models.

Oscar:
So, if you ask someone at OpenAI, what do they think the future will look like? They think it’s just going to be a single model that is pre-trained for everything. And then you just like, what do they call it? GPT-3, right? It’s just a text model that you can write any text query to. And it just gives you something back. So they’re betting on this one big generic model.

Oscar:
Our beta is the opposite, is that … Or maybe not an opposite, but it’s a different take on that paradigm where you use the internal representation of one of these big models and basically it sucks features. And then you use something to train on top of that.

Oscar:
And I think that gives you the best of two worlds because with these deep models, what they’re able to do is take this really high-level unstructured data, natural images, or just natural text. And put them in these relatively low-dimensional embeddings, like 500 flows or something like that. And those are so information dense that once you take those and apply that to your own data, you don’t need very much data at all to find the patterns that you’re looking for in your data. I don’t know if that makes sense?

Mike:
No, that kind of makes sense.

Oscar:
What we do is that we don’t just use one, we use tens or sometimes hundreds of these different networks. So what we do, when you just add one sample is that we literally try like thousand different machine learning models on your data in parallel.

Oscar:
So actually one challenge we have with the site is that it looks so easy. It looks almost like a toy because we made it so easy. It’s like literally, it’s just this some sort of toy platform. But the reality is that we’ve paralyzed the machine learning, we call it AutoML engine, like the automatic machine learning system, to like tries thousands of things at the same time. It just picks the best one for your data. Bam! That’s the one you’re going to have.

Mike:
Right. So instead of you as a developer having to go in and tinker like, well let me try it this way. Let me try it that, let try this one. Let me try that one. It’s like, hey, I’m just going to make all these attempts here.

Oscar:
Yeah. And that’s machine learning in a nutshell. I mean, the machine learning theory is so lacking, that there’s no useful theory around machine learning or deep learning, it’s all trial and error. When someone like me has been doing this for 20 years, all I can do is add some intuition for how to explore that search space of which things to try. So that’s what we’re doing at Nyckel.

Oscar:
Okay, we’re just acknowledging, we’re going to have to try a lot of things. And we’re using a meta layer to figure out which things to try for any particular set of data, to make it cost effective. So that’s our take on it. And I think so far, so far, so good.

Mike:
That’s fantastic. Yeah. And you guys are with Y Combinator now, is that correct?

Oscar:
Yeah, we just concluded the Y Combinator program.

Mike:
You just concluded it, you said?

Oscar:
Yeah. Yeah. So it was the batch started in January and ended in end of March. So we did demo day. We raised seed round, and so on.

Mike:
Was that fun, exciting, interesting experience?

Oscar:
I have mixed feelings about it. I mean, I think it’s worth it because there’s like a rubber stamp. And they give you some money. And it’s also eliminates the unknown unknowns of like, what is it that I don’t know, that I don’t know? That I can’t even ask about, right? And we’re all first-time founders, we’re like, what are those things that we should be worrying about that we don’t even know about? So in that sense, it was very useful to eliminate those. It’s like the bootcamp of running a startup.

Oscar:
And then at the end of the day, I think … We have three co-founders, we’re all in our forties, we’re all like, 20-years experience of being engineers and managers and so on. We were like, yeah, I mean, I think the actual piece of advice from these talks were medium value. Or we could get them from cheaper sources. But it was good to know that there was nothing obvious we were missing.

Oscar:
And also the community is very strong. It’s a big program, right? It’s 400 companies now, two times per year. And all the alumnis are in there and they’ve done a good job of having internal forums. And it [crosstalk 00:13:52].

Mike:
Right. Yeah, that’s got to be one heck of an alumni, one heck of a network to lean into.

Oscar:
Yeah, no, it is. It’s super cool.

Mike:
That’s cool. So you’ve been doing this stuff for like 20 years working in deep learning and ML. What do you think has been the biggest change since you started, what the landscape looks now?

Oscar:
That’s actually a pretty easy question. It’s the paper, Alex Krizhevsky is the grad student who was the first author. And that changed machine learning forever. I mean, I’m sure it’ll change again. So not like eight or 20 years will be another breakthrough, but it was a legitimate breakthrough. And it took like, I’ll say two years before even people in academia got how big it was. And then it feels like every two years, there’s another breakthrough and another breakthrough.

Oscar:
Even these transformer models that came out just a couple of years ago, that was like, it felt like we had plateaued a little bit with a convolutional architecture. And then someone invented this transformer, which is another building block in how you design the neural networks. And that gave another like, invigorated the field again in a very compelling way.

Oscar:
So yeah, no, it’s been cool. I mean, when I choose to go into machine learning, it was a quote unquote, bad career choice. There was no jobs. There no, nothing was working. I lived in Sweden. There was not a single ML company, really. I did it. I was just so fascinated by it. And I got lucky when I did my PhD that someone else came up with this breakthrough. And I was in the right place to understand it and appreciate it and build on it. But that was luck, that was definitely a good time.

Mike:
That’s cool. So you started this company Nyckel, with some other folks. What made you decide to start it? Did you just see a need in the industry? Like the need you described to me seems very real because I know what you’re talking about. Maybe the intimidation level. I mean, it was just three years ago I was at a conference, a tech conference, talking to some guys and they were talking about using, basically rolling their own, using Keras.

Mike:
And they were trying to integrate, they were at a company that sells cell phone insurance. And they were trying to find a way to help reduce fraud. And they were trying to roll a model that would like, they’d be able to take a picture of the screen and this and that. But even then he was talking about rolling their own model and all this stuff. And I’m just thinking, I’m like, man, there’s just so many moving parts to get that up and running. And have it keep being good, you know what I mean?

Oscar:
Yeah. No, it’s insane. There’s a field called ML ops basically that tries to … Like if you’re a machine learning engineer, you do want to turn all the knobs yourself. I’ll send you one of those, designs or diagrams of all the components you need. It’s just ridiculous. And there’s so many traps. It’s not like if you write software, you run it and you get an error if it doesn’t work.

Oscar:
With machine learning, it’s a little bit more devious. You can actually run it, and it’ll compile. But you made some mistake, for example, how you treat your training and test data. So you think your model is much better than it actually is. And then you go to deployment, then it’s just junk, right?So there’s still some really devious best practices that aren’t complicated, but it’s just, it’s yet another thing to think of on top of all the infrastructure layers that you also need to build out.

Mike:
Yeah. No, absolutely. I just think about all that work that goes into trying to build something like that. And I guess here’s my thought, and this is where I get excited about it is that if it can all be reduced to an API call and I can be a software developer, or I can know how to make an API call. If I can figure that out, then I can get access to something where it’s like, there’s this huge body of knowledge and functionality that I can exploit, that I can use just with that single call. To me, that’s pretty darn exciting. And that’s where I feel like it’s all there, but it’s all locked up to these, it’s not necessarily accessible. So that’s to me, it seems like a really fun thing to start exploring.

Oscar:
Yeah, we get a lot from users that start, they’re like, wow, this is … They start with one use case and then they come back with another one. They realize, oh my God, this is so easy for my image use case. Now I’m going to apply to my text or tabular data. Once you realize how quickly you can get up and running and how accurate it is, a lot of times we’re doing better than Azure and Google and Vertex AI models. We have customers that are migrating off those platforms because our accuracy is actually better, let alone how it’s more user friendly. So we’ll see how it goes.

Oscar:
I mean, one of the challenges we have is that it’s such a broad appeal. I think in theory, if not already today, in 10 or 20 years, every single company in the world will use machine learning one way or the other, right? If only to just categorize emails, right? Or invoices, or something. So the challenge we have at Nyckel is that it’s so generic in general that we don’t even know how to market it. So that’s why I’m here today just to talk about it because it’s [crosstalk 00:19:08].

Mike:
Right.

Oscar:
My guess is that every single one of your listener have something that could be solved with machine learning. It’s just a matter of finding those people and helping explain to them that it doesn’t have to be so hard that you think it is, right?

Mike:
Right. So what is the current pay model look like? Let’s say somebody just wants to jump on the website now. Let’s say somebody’s starting a garage business or something like that, so maybe they don’t have a ton of capital, but they want to get into it. What’s it like for them to go from, to scale? What’s it look like?

Oscar:
Yeah. So our pricing model is that you only pay per invoke, every time you actually call the model. So training is free. You can create as many functions as you want. Data storage is free. So you can at the startup plan, which is free, forever free tier, you can invoke 1000 times per month for free. So, 30 times per day. And for a lot of people, that’s enough to get started and convince themselves this is good. And then we have a production tier at 50 bucks per month, that gives you 10,000 included in that price. And then additional, what is it? Half a stent, 0.5 cents per invoke. The more invokes you put in per month, the better price we give you per invoke, basically.

Mike:
Right. Fantastic. Okay. So it’s literally free to get started. Somebody can just go build a model, mess around with some data, play around with it, see what the accuracy’s like and just go. I mean, that’s not bad.

Oscar:
Yeah. And it’s an always-free tier. It’s not like in three months, we’re going to tell you, you have to upgrade. You can sit there if you want and just use it as a hobbyist.

Mike:
Right. Wow, that’s pretty cool. So you’re not only doing Nyckel, but are you also at [Motional 00:20:53]? Is that correct?

Oscar:
No, I was actually, as of two weeks, I’m no longer Motional.

Mike:
Okay. And what were you doing there? I’m not sure what Motional is exactly?

Oscar:
It’s Motional is, it’s self-driving car company. It’s a direct competitor to Waymo and Cruise, that you probably heard of? The brand name Motional is just two years old and they haven’t, it just doesn’t have the name recognition. But it’s the same thing, we’re building a fleet of, or they’re building a fleet of autonomous robo taxis.

Mike:
Okay.

Oscar:
And my job there was the machine learning lead. I mean, I had 100 people actually in my team doing essentially what Nyckel does for you. It’s all the pieces of what Nyckel is, because you need all the pieces to deploy machine learning model. You have the team that finds the piece of data to train on. You have the team that builds all the infrastructure to like permission training, permission inference, manage the data sets, yada, yada. You have a team working on defining the metrics and what to measure, how to measure it. And then you have a big team that actually does the current work modeling, they’re like the machine learning, the core machine learning, if you will. Like trying out different architectures and training and like the data scientists, basically.

Oscar:
It’s a very hard problem. It’s an insane problem. That’s the case where, that’s like as far as you can possibly get from what we’re solving with Nyckel. It’s like the hardest possible problem arguably today. Whereas Nyckel’s trying to solve the simpler, mundane problems that a lot of people have.

Mike:
Right. Yeah. Those simple mundane problems are so common though. And it’s so important to … I mean every room in my home has a light switch, but without a light switch, it would really suck. You know what I’m saying?

Oscar:
Yeah, exactly.

Mike:
And there’s a lot of those situations where I just need a simple yes or no, or whatever. Instead of going through and like yes and no myself and that sucks. That’s cool. Wow.

So you’ve been just doing this hardcore machine learning, deep learning type stuff for quite some time. That’s cool.

Oscar:
Yeah. Yeah.

Mike:
Before we call it, one more thing I wanted to talk about. I saw that you had worked on a bike helmet, an airbag bike helmet? Is that one way to describe it?

Oscar:
Yeah. Yeah. That was my first job out of college. It’s funny because it’s been around for what, 15 years now or something? But it hasn’t hit the American market yet. So every two or three years, someone finds out about it again and like a whole circulation of these videos. Because it’s a crazy thing. It’s a collar you put around your neck with an AI chip in it. And it understands as you’re flying through the air that you’re in a bike accident and then it inflates, a big old hood, helmet type thing around your head, before you hit the ground.

Mike:
Gee, before you hit the …

Oscar:
Right? Yeah. So it was kind of an insane project. But again, I jumped at it. I was actually the first engineer that did founders work to designers. So I had to design the hardware, the sensors. I didn’t do the electronics, I just don’t understand that stuff. But I picked out the sensors and then built the AI system to figure out what’s an accident and what’s not an accident.

Mike:
Wow. Is that a pretty hard thing to determine? Because I was thinking about like, man, when does it know to … I imagine it’s just looking at what, acceleration and how you’re turning and all those types of things? I feel like there’s a lot of modalities of bike riding where it could be misinterpreted. And it’s like …

Oscar:
It’s very hard. I mean, we tried to do an explicit algorithm, that reasons about it. Like, okay, it’s this much acceleration, this much angular velocity in this and that directions. Then yes, it’s an accident. But that was just such a fool’s errand because once you’re in the air and there’s so much noise and chitter going on. So we just went back to machine learning and coded the signals like each vector. And then the problem of course becomes finding training data.

Mike:
College students?

Oscar:
College students were very useful forgetting the non-accident.

Mike:
Put a bunch of bicycles out in front of a bar at night.

Oscar:
Yeah. I mean, it was actually insane. We started using … What would you have done if you needed accident data?

Mike:
Oh, geez. I don’t know. Maybe young people? Well, that’s probably not a good idea. That’s not a good idea. I’m just saying that they don’t have as much skill they’d crash more often. But I mean, because the difference between a fake crash and a real crash, I mean, that’s really hard, right? Because you’re going to be physically, your body’s going to be doing something different during a fake crash than it is during a real crash. So how do you control for that? Oh my gosh.

Mike:
I guess I know one thing I’ve heard, and I don’t know if Motional did it or not, but I’ve heard about the whole self-driving. They actually use data from simulators potentially. So I don’t know if that would even be an option though. I mean, because then you’re talking about trying to simulate a crash to begin with and that’s a hard job.

Oscar:
Right.

Mike:
I don’t know, drunk people. That’s what I’d do, drunk people.

Oscar:
And maybe we should have tried that. But they will also fall differently, I think.

Mike:
That’s true. That’s true.

Oscar:
No, we started with the crash test dummies. That seemed like the right thing to do, but that was actually too expensive because it takes so long. They’re heavy, right? They’re like 80 kilos or something. They’re like a normal sized person and then mounting them on a bike, would rig it up with wires and stuff. It was just such a pain. So we ended up going with stunt professionals and have them wear an actual helmet in addition to the sensors.

Mike:
Oh, okay.

Oscar:
Like a big old motorcycle helmet. And then we would just, it looked like one of those TV shows where they would biking, and we would hide behind some … And then just jam something in their tire, like jam a stick in the front tire without them knowing. Or throw soap on the floor, and they would fall. It was hilarious.

Mike:
What a great first job, man.

Oscar:
But you know what? It actually works. My mom has one. My dad has one. In Sweden, they’re very, very common. Like every one in three bicyclists has one of these.

Mike:
Oh, that’s nice. Well, looked, I mean, having the breeze in your face is part of the fun of riding.

Oscar:
Yeah. And I think it’s changing a little bit, like helmets are becoming more acceptable maybe today. But at least when I was growing up in Sweden, there was no way you were wearing a helmet. Partly because you’re using your bike all the time. You’re biking to work. You’re biking to the park. You’re biking to the grocery store. So then you’re always schlepping this big old helmet around. And your hair is always, if you care about your hair, your hair is always going to be messed up. A lot of people were like, ah, nevermind, I’m just not going to use a helmet. So that’s where it came from, that situation.

Mike:
Right. That makes a lot of sense. Awesome. Well, Oscar, this has been really fun, man. I really enjoyed learning about Nyckel.

Oscar:
Yeah, thanks for having me.

Mike:
Just for everybody out there, if they want to go learn more, where should they go? Just Nyckel website.

Oscar:
Just go to Nyckel.com. And we’re very interested in talking to people. So even if you just listen to this, if you want to chat, you have some questions or ideas or talk about your use case. There’s a little Contact Us button or a chat widget in the bottom. Or you can just send me an email at Oscar@Nyckel.com obviously.

Mike:
Okay. Yeah. And that’s N-Y-C-K-E-L, Nyckel.com. Cool. Fantastic. Oscar, thanks so much. Have a great one, man.

Oscar:
Yeah. Thank you. Bye.

Mike:
See you. Bye.