What are Data Centers and how do we all use them even if we don't use Generative AI?

This is Gulf Coast Life I'm Mike Kiniry. Thanks for joining us. Data centers are all over the news lately because AI companies are building them apparently as fast as they can manage. They take up huge amounts of resources, including electricity and water and hardware and all of the things building huge facilities requires. States and cities and towns are trying to figure out whether they are benefit or a draw on resources that outweighs any benefits. People who live near ones that have been people who live near ones complain of noise and pollution sometimes. But what exactly is a data center? And why do we need so many of them? And why do they consume so much electricity and water and other resources? What kinds of data centers are they? Or are they all doing the same thing, or are there different kinds? It occurred to me that we talk about data centers, like everyone knows what they are and what they do, but that's probably not the case. So today we're having a data centers 101 conversation with Doctor Leandro de Castro. He's founding director of Florida Gulf Coast University's Dendritic Institute, and a full professor of artificial intelligence and data sciences at FGCU. Welcome back to the show, Doctor de Castro. Thank you. Thank you for having me back. It's great to be here. This may sound like a strange place to start, but when did you get your first computer? I'm trying to get a sense of your timeline with computers. Yeah, my first computer was in the late 80s. Okay, so at that time, I was still in high school, first years of high school and then early 90s. When did you first get on the internet as we know it now with the World Wide Web when it first came out, I guess if you had a computer in the 80s. Yeah, exactly. It was around the 90s. And from 1920, 1992 on was I was doing my bachelor's degree. So I was fully online using emails and everything. So we're here to talk about data centers from your perspective, as broadly as you can explain to a listener what a data center is. And I don't necessarily mean what's being built now by AI companies, but just what is the concept of a data center? Well, the concept of a data center is related to a building or a facility in which we are going to store and to maintain a number of computing components. Right. Mainly servers. And their goal is to provide storage and processing for different types of companies. And when you say server people have seen on movies, there'll be a room with racks of things, and inside of those there'll be a hard drive like we have in a computer where you store stuff, but also the CPU, which is doing processing. And so that's what you mean by a server. Yeah, absolutely. Well, actually the definition of a server server is basically a computer, but which is designed to provide processing and storage for different users. Whilst a standard computer, a PC, a laptop is for a single person user, single person use a server is for many people to use simultaneously, right? Did data centers always have the processing side or early on where they mostly about a place to store and retrieve information? No, they always had the processing side as well. If you think about it, what happened was mainly during the year 2000, around the year 2000, most companies used to have their own servers, and this meant that the companies had to now have this computing equipment and to keep them, to maintain them, to cool them internally. And this is very costly and this is very energy inefficient, right. If you compare, for example, the use of a standard server hosted in an enterprise, it's much higher than the power consumption of a server in a data center, right? It's around three times the consumption of a server in a data center. So a data center and the data centers were created to reduce costs of enterprises, and also to make the whole system more efficient. So that would have been around the time that what we think of as the cloud was born with Amazon Web Services and things like that. And just to recap what you said. So before then, every company that needed a server had one in their building somewhere, and they had to have somebody that was there to maintain it, and they had to cool it and do all that. And then Amazon came along and said, we're going to build these giant warehouses full of those things for you, and then you can connect to them. I guess that required high speed internet, right? Yeah, absolutely. And actually, the history of this story with Amazon was slightly different, because what happened with them was that they built this big data centers for themselves to provide their services, their online sales services, and they started realizing that they had a lot of spare time because when when you plan system like this, you have to plan it for the peak hours. So they plan them for the peak seasons, Christmas and other seasons in which they had a lot of sales. And in times when they didn't have that much sale, they had a lot of computing power available and computing storage available. So what they did was they decided, okay, so let's rent this space and this computing power during the times that we are not using them, right. So that's one of the main reasons why they started providing the Amazon Web Services now known as AWS. So like YouTube is something that people will be familiar with. Everything that you connect to on YouTube is held in a data center somewhere, which is filled with racks of these machines. And it's obviously doing processing because whether people realize it or not, when you upload a video to YouTube, it's turning it into something that works for YouTube that everybody. So YouTube would be a great example of a data center for storage and processing. Well, YouTube is actually a user, right? Because the service provided by YouTube, which is the YouTube videos and everything that we have in our YouTube pages, they have to be stored somewhere, right? Google has its own data center, and YouTube is hosted in Google's data center, right? These big, massive data centers are called hyperscalers, and we have a few of them, right? AWS, as we mentioned before, is one of them. Google meta has one of them, XII. And all these big tech companies have their own hyperscalers. And the hyperscalers. How long have those been around? Since about when AWS started scaling up. I mean, you know, because we're talking now and we'll get into what we're talking about now in a in a little bit, but we're talking about, you know, these data centers that take up, you know, hundreds of acres of space. I mean, they're huge. And that that would be the hyperscalers. Well, the definition of hyperscalers usually work around 100MW plus, right? Some of them are bigger than others. And the need for a lot of land, a lot of space is also related with the need of generating its own power, the need of cooling systems and all the infrastructure that you that a data center requires, right? So the more servers you have, the more computing power and computing storage you have in a data center, the more than you tend to need. Cryptocurrency came along and as I understand it, at first people could quote unquote mine cryptocurrency, which we won't get into what that's all about, but it's basically doing a lot of math if I understand it correctly. And then people would build racks of the machines that would do the mining, and then the data center people came along and said, well, we'll do that too. So cryptocurrency before we had AI was a driver in the growth of these things and the need for them, right? Yeah, they certainly had their contribution. But there are not really many numbers saying how impactful the cryptocurrency mining has caused in data center expansion. If if we go for this side of this, this type of reasoning, AI has had a much greater impact than mining cryptocurrency, right? I mean, in fact, in the data center growth process. But the cryptocurrency, I think, would maybe be a bridge because cryptocurrency is when they figured out that you could use what are called graphical processing units, which are designed for video games, mostly originally, and I guess any kind of video editing and things like that. And they do well explain the difference between a CPU, which is the thing inside your computer that's doing the, the, the normal processing and what a GPU is. And then we'll bridge that gap from crypto to AI. Yeah, sure. Well, the CPU is the central processing unit and it's used in the standard computers, the PCs, the laptops and. Our phones. And our phones. Yeah, they are basically targeted at general purpose processing of information. Whilst the GPUs, the graphics processing units, they were created, as you mentioned, for games. But one of their characteristics is that they are very good at numeric processing and they're very good at parallel processing, which are characteristics that are relevant for mining cryptocurrency and also for performing most of the AI maths that we need to generate, to train and to use the models. You've been in the world of AI and data science and machine learning for a long time. When early GPUs came along, were they immediately being harnessed for that, or are you still tied to CPUs before you saw the benefit of these graphical processors? Yeah, well, they took some time until we really realized how much more powerful they were to do the procedure that we need in AI. So and for some time, some companies like Intel, for example, we're trying to develop chips that were more efficient for pro seni types of instructions and maths processing. Right. I don't know how far that has gone, but certainly the GPUs are still much more effective. And what happened to the market was when we really realized how much GPUs were more efficient for this type of analytical processing, the whole market started shifting from CPU to GPUs for AI related applications. And GPUs require more electricity than CPUs. Is that true? Well, they usually do require because of the type of processing that they do and their computing capabilities as well. So people who turn on their TV every day, they hear the news data center, this data center that OpenAI put out ChatGPT about coming up on four years ago. Now, the other companies have come along. Everybody's using AI for different things and talking about it. And all of that requires these data centers filled with servers, filled with CPUs and GPUs. And that's really what we're talking about now, right? Yeah. Yeah, absolutely. And they take up so much energy because of what? Well, they basically have three uses of energy. One of them is running the processor, the processors themselves. The the second one is the cooling of this processors of these processors. Because you need to cool down to, to keep them functioning properly. And the third use is the battery, the backup system that you need for the whole power system. Because if you have power grid shutdown, for example, your computers cannot go off. They have to keep running. So you have these three main uses of power. And they are running 24 hours a day, seven days a week, 365 days a year, which would be different than other kinds of of data centers. These these AI data centers are pretty much on all the time, right? Yeah. But they standard data centers too. They have they have a service level agreement in SLA, usually of 99.9% and above. Right. So the idea is that they never go down. And they also work with a lot of redundancy. Most systems have a three tiered redundancy system. Right. So if one fails you have another one. And if the other one fails you have the third one. Right. And this includes backup. You mentioned earlier when you're talking about hyperscale facilities, you mentioned 100MW. I did I had I had a large language model. Give me some data here. So 100 megawatt continuous power draw data center would be the equivalent of 80 to 100,000 homes. So that's a lot of energy. Yeah, it is indeed the question that I would like or the reflection that I would like to bring to everyone is how much computing power and how much how many different solutions are being provided by all these servers. Because what I see in this debate is, yes, definitely. They consume a lot of energy. They require a lot of water for cooling the processors, but at the same time, they provide us a number of features in a number of solutions that save a lot of energy as well. In many cases, it may save a lot of water. Right. I can give you an example. Let's take the simple case of any smart building, a building in which the elevators are controlled by AI, in which the lighting system is controlled by AI. So everything or a number of features of the building are controlled by AI in such a way that it's much more energy efficient. So this was all generated and this is all controlled by AI. So how much energy are you saving on this end, and how do you compare that with the amount of energy that you have to spend and use to make that AI function in that building? Right. This is the type of investigation that you cannot find many studies, and it's very hard to to build these studies because there is no direct way of computing these things right at finding, okay, what's the exact AI model that is being used there? At what time for how long versus what's the energy that it's saving on the other side? It sounds very complicated. There's a lot of sort of unknowns in it that what that reminded me of is I've often thought of, you know, maybe somebody has done the math on this, but, you know, you don't really get lost anymore when you're driving around because we're using our phones and GPS and a data center is involved there somewhere with that service. And I wonder if you could just tally up at the end of the year how many people didn't wander? Didn't you know? That's another example of how, you know, there may be an overlooked benefit that we don't see. Absolutely. And that's a great example. Right? I remember I was young, traveling with my father. We were we arrived at some polo city, which is, you know, as big and as complicated as New York City. And with this big map in the hand trying to find someplace, we stopped in a gas station. We had to to find some help. We spent a lot of time. We drove around for quite some time until we finally reached where we wanted to reach. Right? So all that is time and energy and fuel being wasted. Yeah, that's that's a great example. These companies, they're spending trillions of dollars and they're not really making that back on the retail end, I guess, you know, from your perspective, how is it going to balance out in the long run with the amount of capital investment versus the amount of return they get on that investment? Because we've got all these sort of squishy examples of benefit. But if they can't monetize that, is that going to be a problem for them at some point it seems. Yeah, I understand that it varies from company to company, right. If you think about Google, for example, Google has its own AI model, Google Gemini, its own AI tools. Google provides a number of other services and many of the services that Google provides. It doesn't monetize from these services. For example, you don't pay to use Gmail. Of course, we all know that we are. The product. Right there are benefiting from our use of Gmail somehow. Exactly. But my point is exactly this, right? Some companies, they can use AI to improve their revenue streams in other areas in such a way that they don't need to focus that much on monetizing AI itself, because they have they have other sources of revenue. But we had we have other companies, for example, ChatGPT or OpenAI, right, that they don't have this business model. They are basically selling the subscription, which we know that in most cases does not pay for their own expenses, but they're thinking about different ways of monetizing. Right. So my point is that maybe the AI market, or at least this general purpose models, is not going to be focused on monetizing the use of the model itself, but on benefiting from the model for other types of revenue streams that they may have. With the early days of the internet. Be an example. I mean, we had the.com boom and then we had the bust and but the internet stayed around and other people figured out how to use that infrastructure. And maybe, you know, the AI infrastructure will just be that heading off into the future. Oh yeah. Absolutely. And it's going to grow significantly. But what we see now is already awesome in terms of what we can do with these models. Every day we find something new, some new type of applications, some new models, some new app being delivered, and it's only going to grow if we think about, for example, computers is still mostly performed by silicone based devices, but when we go into more into quantum computing and other approaches, for example molecular computing, we'll have a jump in AI again, another jump, because when you are at the molecular level, for example, the types of interactions that you have, your capability of storing information, of information, everything runs in parallel. It's a completely different world, right? This will make AI even more powerful than it is today. Might it make it be more efficient in terms of energy and water? Absolutely. I mean, computing at the molecular level, for example, is much more energy efficient because everything is done in parallel at the molecular level. Because one of the big complaints is the environmental side of it is the water use. So it seems like, you know, they're going to have to figure something out if they're going to keep building these things. There's only so much water, right? Well, there are some things being done in this direction. One of the things is related with the technology itself. So we have cooling systems that are much more efficient. Some of them they reuse the water. Some of them use other types of liquids that don't need to be replenished so often. Another thing that many companies are doing, mainly the big companies, is they're finding ways to compensate for the water they use. For example, investing in projects that are related with the protection of water sources and things like that. Right. And many of the hyperscalers, if we look at them currently, they are trying to be water positive in less than 3 to 4 years from now. Right? Some of them are already water positive, right. And of course, I also understand that there is a debate around it because in many cases they are water positive, but not exactly in the place where they are using the water. Right. And in many cases. Like carbon credits or something like that. Yeah, exactly. But at least we see that there is a concern and there is a movement in the direction of, okay, we are using resources, but we are trying to to bring back these resources. I'm pretty much out of time, so I'm going to have to end it there. But you know, these things are not going away. They're giant buildings that are doing tons of processing that generate lots of heat. And we are all in some way probably benefiting from them. Anything else you'd like to add? People like to use this, this number, that one prompt is equivalent to ten Google searches, right? For example. But this is a very abstract number because what is this prompt like? How complex is this prompt? What exactly are you getting out of it. And we can ask the same questions about the search engines. Right. But my point is how much effort can you save with on prompt when compared with one Google search or one being search or whatever? Right. When you do a search, you have to take the result back. You have to look at it. You're probably going to do another search to find more things. Then you have to look at everything to put together to interpret. This is going to cost a lot of time, and maybe with a single prompt you can get everything at once, right. So there is always this balance, and we have to think about the benefit that we get from the user of data centers and from the use of AI. Next, we'll have to do what is a prompt tech show for people who don't even know what that means. That's when you ask a question to a language model listeners. Doctor Leandro de Castro is founding director of Florida Gulf Coast University's Dendritic Institute. He's a full professor of artificial intelligence and data sciences at FTC. Thank you so much for your time. Thank you. Thank you for having me here. If you missed any of the show, you can always hear all of our episodes in their entirety on our website. Or wherever you find podcasts. Our show today was produced by yours truly, with audio production from Pam James and video production by David Oquendo For now, thanks for listening. I'm Mike Kiniry and this is WGCU-FM Fort Myers 90.1 WMKO Marco Island 91.7 FM, NPR for Southwest Florida.