EP 033 Arduino Json Library with Benoit Blanchon
Json Conversation Transcript:
Michael:
Welcome, everybody. I have the pleasure of speaking with Benoit, he is the creator of ArduinoJson, which is an Arduino library that allows you to send and receive packets of Json. Now if you’re wondering who Json is, don’t worry, we’ll talk about that in a moment.
But Benoit thank you so much for being on the show. I appreciate you being here.
Benoit:
Thank you, Mike. I’m very pleased to be here.
Michael:
Cool. So the reason I’m so excited to talk about this topic, is because you the listener are going to be so pumped once you get your hands on this ArduinoJson library, and you realize the power that it’s got available to you. What it basically allows you to do… And Benoit correct me if I’m wrong. It allows you to kind of go out and reach out to different websites that might have information on them, that’s available to them through an API or something like that.
It can allow you to grab that information, get it to your Arduino and then use it in a useful way. Is that somewhat… I don’t know. That’s kind of a pretty rough way of saying it. But Benoit does that sound about right?
Benoit:
Yeah, that’s about right. I can give you a concrete example. If you want to display the weather in Paris on your device, let’s say on an LCD screen, you could connect to open weather map, retrieve the weather forecast for Paris, and pass the information using the library. And the library will help you extract the weather forecast from the response. That’s the job of Arduino JSON.
Michael:
Oh, that’s fantastic. Because that’s the kind of thing so many people want to do. And I think what’s cool is there are so many sources of data out there that are public, that you can just pull out from.
I don’t know, every major website I can think of has an interesting API. Twitter’s got API, they probably got more than one API.
Benoit:
Yeah. GitHub as well, yes.
And while there also cryptocurrencies, you could get the exchange rate for example. I know several projects do that. There is plenty of web APIs that you could use.
Michael:
Right. And one that I have played around with in the past is NASA. NASA has a ton of APIs available. So for example, you could read in the weather for Mars. Mars weather, and stuff like that. There’s just all types of fun stuff, and I’m sure if you’re listening, you’re thinking of some data source out there like, “What could I do if I had that data available to a device.”
And not necessarily just to display. I mean, you could display the information, but maybe you could have your microcontroller take an action based on specific things. It’s limitless really.
But the key part, or the difficulty there for the software developer using an Arduino board is once I’ve got this information stream, how do I parse it? And how do like communicate back and forth logically, all that. But that’s where like ArduinoJson comes in and kind of solves the problem, which is pretty exciting, pretty useful that is.
So Benoit why did you build this tool?
Benoit:
Well, that’s a long story. It was back in 2014, the place where I was working, I also sold drinks to my coworkers and they gave us some tokens in exchange for a can, like a soda or whatever. And I was not handling the money. So someone else was selling the tokens. And at some point, we were about with the tokens, and we thought that maybe a technical solution would be better.
So we had this idea of an electrical wallet, where you identify the person who make the purchase with this badge, the badge that the person uses to enter the building. So I purchase an RFID reader, and I hooked it up to an Arduino Ethernet, and as some coworker did a backend in asp.net. And we communicate through REST API, and where the body of the query and the response is, were just document. And that’s where I started with an Arduino Ethernet, which is basically the same as an Arduino Uno.
Michael:
That’s cool. So you had a need, you’re at somewhere you have a need, and you’re like, “All right. Hey, let’s try to solve it.”
And so that was literally the birth of it, writing the code to make that work, was kind of the birth of the ArduinoJson library?
Benoit:
Yeah, that’s it. I initially started with an existing library which is called ‘aJson’. But at some point it took too much space on the device. I mean not only on the RAM, but more specifically on the Flash, because the program footprint was too big.
So I had to come up with something much smaller, and this is how it started.
Michael:
Okay. And I mean, it seems like its kind of rolled into something, or developed into something that’s pretty impressive because it’s not just the Arduino library, you’ve also got a fantastic website. Is it what? Arduinojson.org, is that correct?
Benoit:
Yes.
Michael:
And on that website, you have not just really good documentation for an Arduino library in my humble opinion, really good documentation. But also you’ve got tools on there that allow people to kind of set up their program to parse appropriately, because sometimes that can be a little bit tricky. It’s one thing to have the library, but man, then to have a tool that helps you set up the actual code you need to write is pretty darn nice. And I am sure that’s part of the reason why the ArduinoJson library has been adopted by so many people because you’ve made it much easier for people to use. That’s a huge bonus.
So before we dive into this though, maybe there’s somebody out there wondering who JSON is. But what we’re talking about is we’re talking about JSON, and JSON’s basically like a standard way of transmitting groups of information over the internet essentially, or from one device to another. I’m not saying it beautifully, but basically it’s a way of sending structured data.
So for example you might have heard of an array, or a list. A list, it’s got values in it. Each value’s separated by a comma or whatever. And if you hand somebody an array, they’re like, “Okay. Hey, I can index each item.” You understand how to work with it.
Well, JSON is sort of like that. It’s just a like data structure, but it’s agnostic to the software it’s using. It’s just got a certain structure that allows you to store information consistently, and send it that way so both the sender and the receiver understand how to… I don’t know, send it and read it in.
I don’t know Benoit, am I kind of getting that right?
Benoit:
Yes. So JSON is a data-interchange format. So that as you say, it’s the sender and the receiver understand and talk the same format. I will describe it as that data format that allows you to encode complex data structures into a simple string, and then you can store the string, or send it over the wire, or over the air.
Michael:
Yeah, absolutely. And there are other formats… And tell me if I’m wrong here, but XML is another example of a data exchange kind of thing. But what I like about JSON is it seems very readable just to the human eye when you’re looking at it. It makes… I don’t know, you look at a piece of JSON and you can kind of make sense of it a little bit, which I think is pretty nice.
And then what you do though, is so you’ve got this JSON, and let’s say you’re working with some website. And we’ve been talking a little bit about APIs. And you’re wondering what an API is, it’s basically a way of a website to kind of expose specific parts of data to a user essentially. But not necessarily to a user through some application on your phone, but through… I don’t know, if you wanted to come in and serve that information in some other way, some other interesting way.
And so these APIs they’re generally going to… And correct me if I’m wrong here. But I mean, basically they’re going to be allowing the easiest way to… In many cases to communicate with these APIs is to use this JSON format.
Benoit:
Yes, exactly. Some of them are proposing you both formats, XML, and JSON. And JSON is usually lighter because the format is more compact. And as you said, it’s more readable.
And on top of that, it’s also easier for the program to encode and to decode. Writing an XML parser is much more complicated. And I don’t think anyone uses XML on small devices like UNO, that would be unreasonable I think.
Michael:
Yeah. And I’m glad you’re bringing up the memory part, because I feel like this is a big part of what you’ve enabled with the ArduinoJson library. And you touched on it a little bit earlier, our micro controllers they don’t have a ton of size.
But let’s say you reach out to some API, and it sends you some ginormous string of information in JSON, or it sends you just a ton of information. And if you’re making that call somehow you’ve got to figure out, well how is it going to fit into the memory space that I actually have?
How do you handle that with our ArduinoJson? Just at a higher level, how does somebody figure that out?
Benoit:
So yes, that’s a huge problem. If I take for example the weather data, it can take several kilobytes of data, and your Arduino Uno only has two kilo kilobytes of RAM. So how do you do that?
So one thing with ArduinoJson is that it stores the complete document in memory, so that could be a problem. There are all other libraries that allows you to read the data, read the JSON document, one pieces after another. But the API is complex, and your program becomes complicated.
So there is one trick that I use in ArduinoJson, is using a filter. So assuming that you are not interested in all the information that is returned by the server. And for example, if you are only interested in the weather temperature, or the pressure, et cetera, you could set up a filter, pass it to ArduinoJson, and it’ll only store in memory the values that you requested. So in the end the memory consumption is very small, it’s just what you requested.
Michael:
I see, all right.
Benoit:
Makes sense?
Michael:
Yeah. No, that totally makes sense. So you are able to kind of parse that information down, or kind of just be very specific about the type of information you’re going to want to get. Because an API, the load it sends back might contain a lot of stuff, but you only want a little piece of that. And so you’re able to just select that small piece.
Benoit:
Exactly.
Michael:
Okay, that makes sense.
And I just want to kind of talk through this a little bit here, so basically the Arduino Json library is kind of handling… And tell me if I’m wrong here, but it’s kind of handling like… Hey, here’s some Json now I’m going to work with it for you, that’s what ArduinoJson is doing for you.
The part about actually getting the Json, if you’re on an ESP32, or an ESP8266, or on an Ethernet, that’s going to be… Actually making the API calls that’s that part, that’s not part of the Json library.
The Json library is, once you’ve got this packet information, now let me help you manage this information in a really straightforward way. Is that correct?
Benoit:
Yes, exactly. It will take the Json string as an input, and give you a document that you can query. You can ask for a specific member with an elegant syntax that is very convenient.
Michael:
Yes. And that was what was really impressive to me. So in some of the examples that you have in the documentation, and I know I’ve said it once before, but the documentation is really good. So if you’re wondering how to use this library go to Arduinojson.org, and just check out the docs, kind of read through them, and look at some of the example code. And you’ve also got some good videos up there.
But some of the examples you go over, is first you take an Arduino board that’s not connected to the internet at all, you make your own string, which is formatted as Json. And then you pass that string into… I’ll call it the quote unquote, the ArduinoJson library. But you’re making an object essentially. You got a piece of code, and from that string…
Yeah, I think it was like a string. You’re just passing that string into it, and it’s going to create, like you said, this Json object like you mentioned. So even though it’s not connected to the internet, you’re still passing this information in. And then the idea is you can pass in this Json string. Well, you can get a JSON string from anywhere. I mean, if you have the Ethernet Shield for example, which I managed to get up and running on this library, and using one of the examples in literally a minute. I just uploaded the code through on the Ethernet Shield, and it just worked. It’s like the example pulls some information from Arduinojson.org. But it was just working super quick. So you can use an Ethernet library, or a Wi-Fi library, or whatever.
And there’s so many of those libraries that are really good at just making these API calls, and stuff like that. That’s already built out for you. And then you just pass it in, and then it gives you this object in your Arduino code, and you can just say, “Hey object, I want to get the temperature.”
It’s kind of set up like a dictionary, I guess. The dictionary is like a key value pair. So the key is like the name of the thing, and the value is the value of the thing. So you might say, “Temperature.” You say, “My JSON doc.” And you pass it the word temperature, and then it gives you the temperature. I mean, it’s literally that straightforward. I don’t know, to me it seems pretty cool. I’ve been impressed.
What type of stuff have you seen people using it for?
Benoit:
So we talk about the weather station, the cryptocurrency stuff. So that’s one part of the job you can do without , which is calling in a web API.
Another thing you can do is provide a REST API for your device. For example the project WLED which is very popular, uses ArduinoJson to pass the queries and code the responses. So you can control an RGB light strip, and control the pattern, the lights, et cetera. So that’s the second thing you can do with Json, which is provide a REST API.
And the third thing people usually do is store the configuration of the device. For example, you could store the credentials for the Wi-Fi, that’s a typical example. At the beginning of the program you load the configuration, and then you can connect to the Wi-Fis, et cetera, and you can save the configuration as soon as it changes. There are some caveats that are covered into the but that’s basically the idea.
Michael:
Wow, that is so cool. Oh, I love the idea of actually setting it up with the RGB. That’s kind of fun, with those strips and stuff.
So is there a part of ArduinoJson that you think is underutilized? Is there anything in the ArduinoJson, where you think, “Man, I don’t think people are quite using this like they could.” Or I don’t know. Is there an unsung hero in here? Are you pretty pleased with how it’s getting used, or any thoughts on that?
Benoit:
We talked about the filtering feature, I think this is underutilized. Maybe people don’t know about it. Also something we didn’t cover ArduinoJson is able to write to any kind of output. It can be a string, a string class, a stream like an internet client, Wi-Fi client, secure, stuff like that.
But you can also create your own writer class that will be able to write to anything. And similarly, you can read from any kind of input whether it’s a string, a stream, or a custom reader. So you can adapt ArduinoJson to any kind of underlying file system or communication protocol.
Michael:
All right, fantastic. So I mean, you could grab information, and write it to a spreadsheet if you felt like it. Just anything like that.
Obviously you got to set that part up, but that’s kind of the idea, right?
Benoit:
Yeah, but it’s really straightforward. You just need to define a class with two members, that’s pretty easy. And as a matter of fact, I saw several users use ArduinoJson on computers, and adapt it to the classic studio file interface. F write, F read, and stuff like that. And it works great.
Michael:
Oh, wow. That is so cool. Yeah, I feel like I have just touched the surface of the library. I can’t wait to dive in and start playing around with some stuff. So I’m excited.
People listening, if they want to learn more, obviously they can check out the website. You’ve also written a book about using this library. It’s Mastering Arduino Json, is that correct?
Benoit:
Yes. Well, that’s a way to support the project, but that’s also a way to learn a lot of stuff. So that’s a book that I sell on Arduinojson.org. In this book you will find the tutorials, you’ll find a refresher on C++, that I call the missing C++ chapter, the missing C++ exactly.
That covers everything that should have been explained by Oso tutorials, but was left out. For example, what is the stack? What is the hip? What’s the difference between the Harvard and von Neumann architectures, and stuff like that. Which seems complicated, but really not that much complicated, and really are important to understand what you do. You will also find a chapter that explain how ArduinoJson is implemented, as well as five case studies of real life project. And I explain every design decisions, and how to make the best use of the library. So not only you will support the project, you will also learn some good stuff hopefully.
Michael:
Awesome. Yeah, that sounds like a great book. I’m going to have to pick it up.
Well Benoit, thank you so much for spending some time with me today. This was fantastic, I’m sure the listeners have really enjoyed it. And again, if you’re out there listening to this, and you want to check this out, or you think it might be cool. Go to Arduinojson.org. The website’s there, the documentation, it’s a really slick website. You won’t regret it.
And I don’t know, I’m pretty pumped. I’m excited to learn more. So Benoit, thank you so much for your time.
Benoit:
Thanks a lot.
