AWS re:Invent 2023: Generative AI Essentials for Decision-Makers - Use Cases, AWS Services, and Implementation Strategies

Name: AWS re:Invent 2023: Generative AI Essentials for Decision-Makers - Use Cases, AWS Services, and Implementation Strategies
Uploaded: Nov 22, 2024

Transcript

Auto Scroll On

A

So welcome. Got a lot to cover today. So let's get started. I'm Mark Trin Caro. I'm a senior technical instructor with AWS. Been working in the AI/ML space for about eight years now. Very excited to be talking with you all today about gender to AI for decision makers, decision maker. You know, come in a lot of different forms. Maybe you're an executive looking to understand gender to AI and how you might incorporate that into your business or you're starting a project and just want some insights there or looking to upscale your workforce or yourself on generative AI. And that's gonna be a lot of the focus here today. So we'll start with an introduction to generative AI. We'll move in cover some business use cases for Gen AI. And then we'll go over some technical foundations and terminology and the purpose of that is really just uh we're not gonna give you a collegiate level lecture or anything like that, but go over some key concepts in terminology so that you're able to communicate effectively with business and technical stakeholders working within your company. We'll go into planning a generative AI project, some considerations there as well as how do we evaluate not just the benefits but the risk and potential mitigations that we might employ and then closing things out with building a generative a ire organization. So let's start with an intro to generative AI and before we get into that, it's important to cover machine learning, have a basic understanding of that. And so at its core, machine learning is where we, we've got some data set and we're training a Model on that data set to make predictions on some new unseen data. Now, rather than a classical programming approach where a developer maybe explicitly to find some rules with machine learning, we provide the data and machine learning is training a computer, a Model to recognize patterns in that data, to pick up the patterns and signals in that. So it can make predictions on new unseen data. So taking maybe predicting credit card fraud, for example, example of maybe supervised machine learning where I got a big data set, maybe years of credit card transactions. And I've got those labeled as this is a valid transaction and this is a fraudulent one. And so as opposed to explicitly defining rules, I can take that labeled training dataset, feed it into a traditional machine learning Model and develop some sort of classifier that's able to pick up the patterns and signals hidden in that data to be able to determine if a new transaction is fraudulent or not. And AI M not new to us here at Amazon. We got a rich history over 20 years in that space and we're just getting started. It's a big part of what we do, our past, our present and our future back in 20 or 2001, introducing, you know, personalized recommendation engines on the ecommerce side of the house, things like Amazon, Alexa, as well as even in our fulfillment centers, you know, how do those work? Robots navigate and optimize the route planning, predictive forecasting. You know, what type of inventory, where do we need it? How much? And even most recently, some things in the generative AI space like code whisper and Amazon bedrock. If you're interested in those topics, we got a ton of sessions for you this week, encourage you to check some of those out. And before we dive in, we've probably heard some of these terms before, you know, we see artificial intelligence, machine learning, deep learning, all these tight terms in the headlines. Let's unpack and broadly define them here before we get started. So we'll start with artificial intelligence. We can think of that as almost like the umbrella blank blank blanket term that we use to describe systems that can think or act or reason or do tasks that previously required human judgment. Now, we covered what is machine learning, taking that data, using it to train models to recognize patterns in that and make predictions on new data. A subset of artificial intelligence. And then deep learning. And when you think of deep learning as a specialized type of machine learning with deep learning, really, it's the usage of artificial neural networks. And that really stems from the biological neural networks in our brain. And with those neural networks, we found that we could tackle problems even more complex than traditional machine learning approaches might be able to solve, leading to advances in computer vision object detection, natural language processing those sort of task. And then we can think of generative AI as a subset of deep learning when it falls within that overarching discipline of artificial intelligence. And the keyword here being generative and so rather than simply analyzing and classifying data, maybe like we see with traditional machine learning models, generative AI is powered by massive massive models that are pretrained on internet scale data sets. And we'll talk a little bit about those foundation models here in just a bit. And unlike traditional machine learning models that are more narrow focused on a particular task, like identifying fraudulent transactions, they've been trained on that data and they might be good in that specific use case or domain, but I might need a completely separate Model to do different task. Whereas generative AI can be used, the same foundation Model can be used to perform multiple tasks, you went backwards there. And so why now, why are we seeing it? Big buzzword right now? We're seeing a lot of hype around generative AI and it's built on deep learning and the theory for deep learning really goes back decades. So why are we seeing exponential growth in this space in recent years? And it didn't come overnight? But some key factors that led to that were advancements in neural network architectures, specifically the introduction of uh generative adversarial networks or games as well as the transformer architecture. Uh back in 2017, we'll talk on that a little bit later here today. So some advances in the neural network architectures, as well as access to massive massive massive amounts of data, internet, scale data, publicly available data and all different forms, text images, other forms and then also access to massive amounts of specialized compute a lot of those GP U backed instance types and specialized hardware accelerators that are able to run and train these massive models and perform a lot of that complex math calculations behind the scenes. And then also investment not just in the compute but willingness to go big large teams of researchers and that compute infrastructure there. It was a kind of defining generative AI. We've talked about artificial intelligence, machine learning, deep learning and introduction to generative AI. We're seeing it being used to improve customer experiences, uh boost employee productivity and drive business value across a lot of different other use cases like we'll look up and again, unlike traditional machine learning goes beyond simply recognizing patterns, generative AI models can be used to create entirely new content, conversations, stories, images, even code keyword there being generative in nature. And those models are pretrained on massive collections of data like we'll look at in just a bit and I need to stop hitting back on that side. So here how many of y'all have experimented with any form of a AI powered coding assistant? Handful? All right, if you have it, I encourage you swing by um some of our spotlight lab rooms here during the reinvent, we got some cool labs. One even on code whisper like I'm gonna briefly talk about Amazon code whisper is a code completion tool created by Amazon using generative AI trained on lots of public code. Um various SDKS that you might use within AWS like Bodo three and use generative AI to help developers write code can automatically complete code blocks of code. You can even just give a comment and have code whisper, take that comment and generate a function definition for you. So it can boost that developer productivity. Almost 10 X kind of that productivity, but also can be used to help ensure that you're building secure, safe applications as well. So it can help identify um problems during code review, things like here where you see with PI I data and recommendations surrounding how we might handle that. And Amazon ran a productivity kind of challenge during preview of uh code whisper and found that participants were 27% more likely to complete the task successfully. And out of those who did 57% faster than those not using code whisper. And so it's been trained again on massive public code repositories, multiple programming languages haven't checked it out, encourage you to check it out. We'll be doing some questions towards the end. Um, so hang on to that and we'll, we'll, we'll tackle it and also code whisper. We got some self-paced labs. I encourage you to go check out play around with yourself and we're seeing generative AI being used to improve customer experience, like I said, boost employee productivity. And according to Goldman Sachs research JD AI is forecasted to increase global GDP by over $7 trillion in the next 10 years. So let's look at some examples of how customers are using generative AI one of those customer experiences, you know, personalized chat bots, things like that. How many of you enjoy calling into a call center and being put on hold and may being handed off to different agents to get to the solution to your problem? Probably none of us, right? Um So we can have, you know, personalized virtual assistant that can help uh with customer experiences. Maybe you've got questions about a return for an item rather than sitting on hold, we can serve those customers um in a self serve fashion with generative AI and not only does that boost the customer experience but from a business perspective can drastically reduce all that money spent on call centers and handling these customer support kind of task in more traditional ways. And it doesn't necessarily replace, you know, your customer support agents, generative AI could be used to do some initial triaging might be integrated with some of your backend systems to pull back information on what items did this customer recently order problems they've had in the past and the such and then we can even use it um to capture call logs. So maybe you have years worth of call logs from a customer support perspective to be able to make sense of all that data kind of in that raw textual form. Summarize maybe what key reoccurring customer problems might be and influence your business to uh maybe focus on addressing some of those. And then employee productivity. A lot of times we might have tons of different wikis and internal documents and other sources of information. But we spend so much time just figuring out where that resides and how to get to it and maybe asking other members of our team. We're seeing advances in generative AI for intelligent search kind of question and answer as well. So being able to get that information quickly to decision makers, other pieces of your organization, we're gonna look at some examples of content creation, not just text, text summarization, um producing compelling narratives but could be images as well, audio or visual. And then we saw code generation with Amazon code whisper as well drastically can help improve that developer experience. I can speak firsthand of this. A lot of times I would, you know, have to break my my flow and move out of my ID and then go search the internet and read lots of other people's problems and try to get to the bottom of what I'm trying to solve. Whereas code whisper meets developers where the most productive directly in that ID. So you're not jumping around breaking that workflow. And then from a creativity perspective, we're seeing a lot of advances here across music, art, images, animations, you got content generation as part of your business, likely something you're already considering or gonna want to consider and then business operations as a whole. So document processing at scale kind of repla uh replacing some of the more manual or tedious task, being able to do that at scale in a streamlined automated manner, maintenance assistance. So maybe you've got technicians on a manufacturing floor and they need to diagnose certain problems. We could tap into some of the maybe IoT sensor data of how those equipment is operating as well as standard operating procedures, manuals and the such. So we can quickly get some of that information to people faster visual inspection. And even we're seeing synthetic training data creation, which generative AI a lot of times when you're training machine learning Model, you need training data, but it's not always readily accessible at scale for us and sort of seeing some advances there as well. And looking at some industryspecific use cases for genitive AI and healthcare, we got tools like AWS health scribe that can automatically generate clinical notes by analyzing that patient provider conversations during that Hipaa compliant and all the such there. And then my fiance is actually a nurse practitioner. I can speak firsthand to this. She spends an enormous amount of time on this task. Clinical nodes putting all that into the system and the such. And it can also streamline documentation processes and maybe even be used to generate personal uh visit kind of recaps for the the patient there, summarizing key things that were discussed. So you're not just scribbling, taking notes while you're talking to your doctor there. And then from a life sciences industry, we're seeing advances in drug discovery leading to new therapeutics, looking at molecular structures of drugs and kind of completely reimagining the process that we would go through to discover these new molecular compounds and even in protein folding as well, leading to different amino acid sequences and creating new therapeutics and developing, you know, new cures or things within the life sciences domain and in financial services fraud detection mechanism. So going past just that traditional supervised machine learning type approach that I used earlier, I got a big label data set and here's 10 million examples of of legitimate transactions and a couple of 1000 of fraudulent, we can take that a step further and gender debate. I can even be used to create synthetic data sets that help speed up the process of identifying those bad actors or criminal rings because they're also kind of always changing their patterns and things like that. And so we need to be able to keep pace with that portfolio management looking not just at, you know, technical factors from a technical analysis perspective, but holistic portfolio management um capturing information maybe in earnings reports, other textual things out there, news, what's the sentiment trending like and the such and then a couple other industries here manufacturing a big one as well. I used kind of the the maintenance assistant example earlier of conversational agents maybe that are trained on maintenance manuals and all the standard operating procedures for equipment and then maintenance notes over time, you know, who repaired what and when and maybe even pulling in some some streaming type IoT data of health performance of those. And so it can really be used to optimize some of those automated factory floors, other things like that process optimization improve just the efficiency, discover hidden insights there, reduce cost, minimize waste inside of retail, huge, huge industry. We're seeing a lot of usage and generative AI here how many of you ordered something and it came and maybe it just didn't look or or or feel or fit like it, it looked online when you ordered it not a pleasant experience. And then you also got to return that item and the such. And so we're seeing things like virtual try-ons. And so you can actually kinda see, not just picturing yourself in that shirt, but what does it feel like or what do these new shoes feel like and get some more immersive experiences for your customers and tailor that um for them as well, product review, summaries. And we'll look at an example in just a bit of just that maybe you've got, you know, thousands of different products in your, in your inventory and you need to develop a copy for your websites and other things that can create compelling narratives tailored to specific customer interest. And then in media and entertainment industry content generation, huge, huge. Um You can ask even, you know, scriptwriters, things like that, you can ask generative AI to create a script and give it some sort of tone and persona and things like that. It can help accelerate that creative process be almost like a brainstorming assistant, virtual reality. We're seeing a lot of advances in news generation newsletters, articles, blogs. You can actually use generative AI to streamline that tailor newsletters for different sectors of your customer base there. So we went over a handful of use cases at a high level. We got a lot to pack in today and I want to leave some time for questions at the end. Um We'll work through a couple more here in just a bit. But before we do, I want to speak towards gender of AI services on AWS. And this isn't an exhaustive list. You're gonna see other services. And this week during reinvent expect a lot of announcements likely for advances in gender to AI and other services. But if we start at the bottom layer, we need compute, we need compute for our generative a workloads. And AWS offers a wide range of compute, not just those CPU backed instance types, but GP U backed instance types with different hardware accelerators for deep learning. And so you need compute really at two major times in in that life cycle, you need a lot of compute to train the Model and then you also need some compute to host and serve that Model for inference or making predictions for traditional ML or generating new content with Gen AI. And so when we think about training models, NWS has trainium chip sets custom built from the ground up for training large neural networks, deep learning type workloads. And we also have Inferentia chip sets, purpose built from the ground up for ML inference at scale, not just performance but significant cost savings there. And another thing to be cognizant of is the beauty of the cloud, probably all familiar with it. But the elasticity of the cloud we can grab more compute when we need it, tear it down when we don't. So we don't need to pay for a lot of excess capacity that might be used for certain portions. But we can scale elastic elastically as needed. And then in the middle Amazon SageMaker and so customers can use Sagemaker sage makers manage service. So it takes away a lot of that operational overhead of managing the underlying compute and other things like that for you can be used by your data scientists. ML Engineers. From an end to end perspective, you can use it for data preprocessing Model, training, tuning deployment even onwards to monitoring. We got some other sessions specific to sagemaker if you're interested in that. And at the top, Amazon bedrock. And so this allows customers to access foundation models as a service. We kind of briefly touched on that term, but we're gonna unpack what a foundation Model is in just a bit. This is extremely great because the cost of training, pretraining and building your own foundation Model from scratch. There is no easy endeavor cost millions of dollars and maybe months of compute to do so. Why do that from scratch when you can leverage those foundation models as a service within bedrock and bedrock is serveless in nature. So you can just quickly get access to these models via API calls. You can even do fine tuning, tailoring and customizing of some of those base models within bedrock and none of that data is shared with third party Model providers are used to improve performance of that base Model because privacy of that data, especially if you're fine tuning on maybe some uh corporate data sets, other intellectual property property, you wanna make sure you're cognizant of that. So bedrock lets you customize Finetune models with your own data and you don't have to worry about any of that underlying infrastructure. Sagemaker also provides sagemaker jumpstarts where you can get up and running with a lot of those popular foundation models from Amazon like Amazon Titan, as well as um Clod AI 21 labs, handful of others there. So we've talked about what generative AI is and looked at some some common use cases and how customers are using generative AI with AWS. Now let's walk through an example scenario. Imagine we're a shoe company, we're a big shoe company and we're launching a new line of shoes, some new walking shoes. So let's take a look at how generative AI can be used across various stages in this product release. Starting off. Maybe our product team has done a lot of extensive research on market data about this new shoe that we might be introducing. And in order to move forward, we gotta present that to executives, we gotta get their sign off and approval and executives time is precious. They're busy and so generative AI can help produce concise and formative summaries, taking that large corpus of data and enable executives to stay informed about what they need to know without spending tons of time reading lengthy reports and trying to part uh digest a lot of that complex information. So here we got an example of using a prompt for a Gen AI Model providing, you know our market research report and saying, can you generate a detailed market research report for an executive presentation and then we can see the output there. And from that, we could even ask further questions, customize it. Say can you make it shorter, more concise or go into detail here? And then moving along great news, we got approval. We're gonna launch this new line of shoes. There was a a market fit for that, but we got a tight timeline. Leadership wants us to to launch these in about a week. So we need some content generation. Our web developers are saying, hey, we need to add this to our catalog, give us some web content. And so genitive AI is a game changer when it comes to creating compelled uh compelling content tailored descriptions of maybe your products for website copy streamlines that creative process and especially we're looking at just one new shoe line here. But what if you got thousands of products? Tens of thousands, you can see where the benefits here in productivity come into play. So we give it a prompt of write a product description of a shoe that's good for walking around London and has some of these material attributes. And then the output is we get a much tailored um crafting of that narrative that we can use for our website copy and another example of content generation. So we got our website copy, it's been updated our inventory, but we need to generate some buzz, we need some social media and marketing campaigns. And so janitor AI can be used for the content generation here. And we, and we'll talk about context coming up. If you notice this prompt, we say write a product announcement for social media based on the previous details. And so during a session with one of these models, we can maintain context or understanding of what's been discussed or asked about previously. And also we can use the same foundation or base Model for a lot of these tasks. We don't necessarily need separate unique models like we might with traditional ML and then not just text images as well. We got that tight timeline. Time is chronicle. We're launching this in London less than a week and we got some stock photos of this new shoe, but we don't have the time and budget to fly out to London and generate some tailored images and take, you know, professional photo shoots out there in London. So not just textbook content generation in the form of images. And this can streamline the pace at which that's developed, reduce your operational cost, a ton and maybe our launch is major success in London. And then we want to expand to more global markets. We can use the same and tailor images for markets around the globe or specific segments of your customer base. And then we saw code generation earlier with things like Amazon code whisper. So maybe we've launched the shoe line and we've gathered some sentiment analysis, maybe click stream data on websites, other things. And we want to make sense of it all. In order to do that, we need to get it to an S3 bucket. So maybe our developer uses a code whisper or intelligent coding assistant. And here we can see just based on the comment, create a Python script to upload files to S3. We can actually use code whisper to generate entire functions blocks of code, which is pretty cool. I use it very commonly. A lot of times I use Bodo three SDK with AWS and it's handy for just sting out some comments and, and and streamlining that process there and then things are going great. But we want to improve that customer experience as well. And so earlier, we touched on question and answer bots, chat bots, AI powered virtual assistants. And so from a customer standpoint, like we talked on earlier, this is awesome. You don't have to sit on hold for 40 minutes and then get transferred to another agent to get the answer to your question. A lot of customers prefer to get that quickly via digital self-service channels. And from a business standpoint, we can also reduce a lot of that operational cost that goes into call center time and the expenses to, to field customers via those channels there. And so here we could maybe fine tune that Model like we'll talk about a little bit later on product lines within our business and maybe allow access to some backend databases with uh order information, customer information and so forth. So we can actually get domain specific query responses. The customer comes in and says I want some assistance finding the tracking number for my most recent order. And then from that, we can use generative AI as well as integrations to some of that, maybe domains specific data behind the scenes to come back and say, hey saw you ordered the shoe on this date. The delivery date is tomorrow and here's a tracking number for your reference and keep hitting back arrow. All right. So we've looked at what is generative AI kind of unpack some of that terminology like AI/ML deep learning and generative AI and where that fits into the equation, we looked at some use cases. Now, let's talk about some of the technical foundations and terminology here. And the goal of this really is to give you a better understanding of how gender to AI works and also empower you to have more effective communication with various business or technical stakeholders that might be involved on your gender to AI projects. And we got a tight timeline today. So unfortunately, I'm not gonna be able to give you a deep collegiate lecture here on the topic, but we're gonna go over some key terminology starting with F MS or foundation models. We touched on that earlier. This is kind of um what is the base for a lot of our gender of AI and they're pretrained on a large, large, large amount of data and I can't hit home the massive scale of data that these foundation models are trained on. We're talking internet scale data with billions of parameters a lot of times. And these foundation models and what makes them unique is unlike traditional ML models where we have very narrow focused use cases and I might need 5, 1015 different models to do different task. The same foundation Model can be adapted to a wide range of task. And we looked at some of those like content summarization, content generation and it's not just across text modalities, images, other modalities as well, code and those intelligent AI assistant or question and answer bots. So some examples of foundation models, Amazon Titan Model, stable diffusion, Llama two clod a whole lot of others out there. And so how are these foundation models created that pretraining on that massive scale of data? What if I ask you to learn everything on the internet? This is a very hypothetical stake here. How would you do it? Maybe you got, let's say 6 billion pages on the web, maybe you're a speed reader. You don't need any breaks. 60 seconds a page. We're talking about 6 billion hours of time, not gonna be possible, but a foundation Model can do this in a matter of months. Currently cost millions of dollars to develop a foundation Model from scratch. But the nice thing is much faster and cheaper for your developers, your organization to use these pretrained foundation models rather than training their own unique ones from scratch. So, pretraining this foundation Model requires massive amount of unlabeled data tax damages, audio and large large scale training infrastructure. A lot of that compute there and that leads to models with billions of parameters like we probably have seen in the in the headlines today. And how does all this data get processed? Well, comes in the form of transformers, transformers, we can think of almost the powerhouse of generative AI very famous paper from 2017 called attention is all you need. If you're looking to dive deep on this topic, it's a specific type of architecture that's used within gen AI models, it's kind of like the brain of the system. It allows to understand and generate complex patterns, languages, images, other datasets and transformers are able to pay attention to multiple things at once. Hence the name of that attention is all you need. So if we take an example, I went to the bank last night to get some money for the casino and now that ship is anchored on the bank, same word, completely different meanings depending on the context in which they're using. And so transformers are able to retain and use that information, not just of the word, but its context and its larger string of text, the meaning and the position. So let's use this example to illustrate how those transformer models work. We asked the J AI Model to complete the sentence puppy is a dog as kitten is too probably all know. But let's use it for the sake of example, machine learning models don't take kindly to text. They like numbers, a lot of math going on a lot of matrix operations. And so before passing that text to our Model to process, we need to tokenize that text. So tokens are just parts of that overarching block of text. They can be words, phrases, individual characters, periods and the such they give a standardization of that input data that makes it easier for models to process. So we see that input of a puppy as a dog, as a kitten is to and we see an orange there, the tokens being created. And what about those embedding down below? And that encoding. So in generative AI embeddings are the magic behind language understanding, they transform words models don't really care about those strings of text like math, but they transform them into meaningful vectors that machines can Comprehend. So word embedding is an in dimensional vector that represents that word. And so here, for just example's sake, we got a little bit of a graph and you can see like cat and feline being kind of closely together whereas canine and puppy and young there, so we can think of word embeddings as the bridge between how we use words and how machines interpret and respond to them. So after the tokenization transformer then encodes those tokens to those n dimensional vectors, we call embeddings and then a decoder comes into play. So once all those tokens have been encoded, transformers decoder uses that vector representation to predict the requested output or generator predict that next token. And so it has built in mechanisms in that transformer architect architecture to focus on multiple things at once different parts of the input and guess the matching output. And a lot of complex mathematical techniques that are used behind the scenes to evaluate different options like cat or bird or bear or human depended on the context there. And so our in our scenario decoder would associate that puppy is the dog as kitten is to cat. So the Model outputs that prediction and generates the next word in that sentence. So to reiterate important point about transformer models compared to a lot of their predecessors is they're able to paralyze do a lot of compute in a paralyzed fashion and process where it's not just sequential sequentially, but in parallel all the input tokens at once during the life cycle there. And then context final co uh topic to cover here and some of that foundational knowledge and terminology, context. And that's key context is the text and that active conversation you're having, it does not persist and it has a maximum size. It's finite. We've seen different approaches to context windows and sliding context windows. They can vary in size but they're finite in nature. But context is key. Let's use an example here. How does context work? So we give a prompt. Uh what's the best place in Seattle to visit? Model? Comes back with a response. The Columbia Center offers breathtaking views of the city skyline and then we come back with an additional prompt. So multi turn dialogue, but here in the same session, will this be fun for Children? What is this? How does our transform our architecture, our Model able to understand what this refers to in the current context? Are we talking about a specific place? Seattle, just my visit. And so the Model is here correctly decided that Columbia Center is what this is referring to. So that's maintaining that context, we can have conversations with these AI models. And as long as we have that same session, we can maintain context of customers calling in about a specific order and then we fix that issue and they ask about something else. We don't need to reestablish and figure out what was talked about before. So that's a little bit of a, a quick high level overview on some of the key terminology and a little bit of how these gender to AI models work. Let's talk about planning a gender of AI project likely to focus of why a lot of y'all might be here today. And so we're gonna break this discussion into four different steps with any project. We need to define the scope, especially important with generative AI nailing down a scope and prioritizing efforts too. What do we focus on? First, how do we allocate our time funding and resources? And then there's a handful of models out there. How do we select an appropriate Model based on our use case? And again, we don't necessarily need to train that foundation Model ourselves from scratch. It's a very costly endeavor, millions of dollars months of compute. There's a lot of those open source foundation models that we can take and make selections for that might be appropriate for our use case. And then we can adapt that Model further tailor it using different techniques or relevant data. We'll briefly touch on prompt engineering and finetuning here. Today. There's some great sessions at Reinvent that'll go much, much deeper into some of these topics. But we can actually take that foundation Model and maybe adapt or tailor it to our specific use case and then using the Model, not just click a button, throw it in production, but have we made sure that we've evaluated risk concerns, ethical concerns. Do we have proper mitigation strategies in place for that? How are we gonna, what's our intended usage? How are we gonna integrate that into our application? And even more importantly, feedback loops, extremely important. You wanna know how that's operating and maybe use insights over time to further fine tune or tailor that. So it sounds simple, defining the scope, but it's a critical step in really any project but especially your genitive AI projects asking yourselves, you know, do our customers want this and this might be your external customers, internal customers, your own employees or workforce? What is your target audience? What are your desired outcomes? And how are you gonna measure success on those? What's your success criteria? Because we don't have any of that in place at the start of our project? Great. We've released this new fancy uh technology. But do we have any understanding of, is it delivering value to our business? Is it operating as we expect it to? And then can our organization do this? What's the level of difficulty cost funding any technical challenges, skill sets that we might require and then should our organization do this? So what's the potential business value, revenue return on investment? What is the competitors doing in that space? And then not all use cases are created equal. It's important to access, you know, both short term and long term impacts that different solutions might provide as well as their implementation timelines. How do you go about prioritizing things? Can you start multiple solutions in parallel? Ideally, that'd be great. And maybe you can, maybe you wanna prioritize certain ones. So here we'll consider a couple of examples, maybe an easy win, like uh tapping into Amazon code whisper to improve developer productivity with that generative AI powered coding assistant. That's a fairly easy one. Not much you have to do there really just maybe some training surrounding that and encouragement to your employees. And we'll talk about building an Gen AI ready organization a little bit later. That's a fairly easy one versus other use cases might require some more robust planning and a little bit longer implementation time. And we're looking to improve that customer experience with that Gen AI powered Chatbot or virtual assistant. And we want to reduce all that call volume to our call centers a little bit more planning, maybe a longer horizon here. So once we define that scope, next, I need to select a Model because again, you don't need to create these foundation models from scratch. It's extremely difficult costly endeavor. So selecting what Model might work and with all new capabilities kind of being continuously um coming out in the Gen AI space, it's extremely important to have some form of a framework to evaluate which Model and level of customization that might be right for you or your use case. So we'll use that AI powered virtual system or Chatbot example, we touched on asking ourselves, are these questions from our customers? Just gonna be general questions or are they gonna be more focused and domain specific? So I got options, I can use one of those pretrained foundation models out of the box. And that's suitable for general task. Maybe when time is of the essence and customization is minimal there. But then for certain use cases, maybe I want to be able to actually have domain specific responses for my customers and my products. And so I could fine tune an existing pretrain foundation Model and that gives us the flexibility to customize and tailor that Model and the outputs specific to your needs. So this might be necessary when you have more complex data sets or you're looking for more complex kind of question and answer scenarios in your own domain specific data and knowledge there. It's gonna require a little bit more computational resources to do some of that finetuning though not nearly the amount of compute that goes into creating that foundation Model. So f finetuning is not creating one from scratch. We're just kind of tailoring it, customizing it to meet domain specific needs there. So a little bit extra compute some time, some resources and some skill sets to do that. And what approach is right, try to work backwards from the solution you're trying to solve where will the data come from? What level of customization might I require? And so maybe here in this Chatbot example, maybe we're looking to just use a pretrained foundation Model, but we can feed it some additional context down the road when we're using it approaches like rag or retrieval augmented generation and then adapting our Model a handful of different ways that you can customize and tailor the output depending on your needs and use case there much more than we're gonna cover today. But let's cover two popular methods. Prompt engineering and fine tuning with that Chatbot example there. So then prompt engineering, really what this is is just the process of defining and refining your prompts or inputs in order to have that Model produce the output more specific to the types that you, that suits your needs. And so maybe you've got, you know, meeting minutes and you wanna summarize uh or you've got some text from a meeting and you want to summarize that into meeting minutes. I can just out of the box say here's a bunch of notes from the meeting. Can you summarize this for meeting minutes? But maybe you want some standardization to that the way you do that in your business or your customers expect. And so with prompt engineering, instead of just providing that single input, we could give a couple of examples as part of that input and say here's some notes from a meeting and an example output of some meeting minutes that could be uh so zero shot. Really? I just come and ask what I wanna ask one shot, maybe I give an example or a multi shot there. So maybe a couple of examples and then providing that as part of our input. So we're not fine tuning the Model. We're just kind of doing some tweaks to the input or prompt that we provide that can help produce outputs more suitable to our needs. Whereas finetuning, that's, we can think of it as a continuation of that pretraining that went into creating that foundation or base Model. And it creates a new specialized Model, but it requires some compute like we looked at and some very high quality labeled data, make sure you have at least 500 examples of high quality labeled data that meet your needs or whatever use case you're looking to accomplish there ideally, you know, much more. But this would be an example of maybe you've got those call center logs and you want them standardized and you want the structured output of this is the problem, the resolution, the triage process. And so you could fine tune a Model with some 5000, couple 1000 examples that are very, very high quality labeled. So here's an example of some um call center chat and here's the summary and here's how I want that structured and the such and so fine tuning actually, you change not all the parameters but some of the parameters of the Model to create a new Model specific for your use case. Now, the cost can add up quickly here if you're working with large, large models. So earlier, as part of that Model selection process, different models come in different shapes, flavors, we got 10 bill parameters, 70 bill 100 bill. So depending on the complexity you might need, that might be a factor into which choice that you go with. And then step four use the Model. But this isn't just as simple as click a button and ship it off to production. You need to have a plan in place about that intended usage. And hopefully you got this up front in step one. When you're defining the scope, what is your target audience, the intended usage? What's your success criteria? How are you going to measure the effectiveness of that and then making sure that you've addressed responsible AI considerations and have a plan in place for continued monitoring. So you can actually see how the solution is working in the world and monitoring, not just to monitor that, but to collect data and use that maybe as a feedback loop, maybe use it for longer term, maybe go back and do some additional fine tuning on a larger dataset that you've collected over time. And a lot of the same ML ops principles that you would have with traditional ML models hold true here, all those same best practices and in fact, kind of a term for it FM ops helping ensure that you get your models to production, but also keep them there, keep them aligned with your business goals. And so do you have a plan to collect feedback from users as well? And this can be extremely important and creation of some of these foundation models. We're seeing reinforcement learning with human feedback or RLHF. So it's extremely important to be capturing that some of that might be, you know, producing two different examples and having users select which one they prefer and that can be used and helped to improve that Model over time. And how are you tracking changes to that pretrained Model so that you can then retrain or fine tune using that reinforcement learning with human feedback or other approaches. So four different steps, we can think of going into planning a generative AI project encourage you. Um Biggest thing I could say is scope upfront. Don't just jump into something because it's cool and there's a lot of excitement about it. You need to have a clear well-defined scope before you embark on any of these endeavors. And it's not gonna happen overnight too. We talked about prioritizing different use cases and so take steps, build that culture of gender of AI and we're gonna look at that in just a bit uh coming up. So we've seen generative AI and its benefits, but it's also important to recognize potential risk or problems that might arise. So for any of these cases, first, ask yourself is generative AI appropriate for my problem or a task, not always the right solution, assuming it is we need to take into account the risk of using it and whether or not those can be mitigated. So some different considerations here across fairness and privacy, fairness, making sure that, you know, we don't have any unintended consequences for certain groups of people. Uh you know, especially in highly regulated industries like lending, um loan approval, things like that from a regulatory compliance. And this becomes consumable harder than what we do for fairness with traditional ML models with a traditional ML Model to predict credit card fraud detection or loan approval. I can ensure that the training data set I used is relevant and has examples of those different demographics and so that I'm not introducing bias into that Model, whereas with the generative AI Model is a little bit more complex. So something that should definitely be on your radar a little bit harder to define measure and enforce, but something you should be um at the top of your mind there and then privacy big concern here, especially um you know, making sure that these models don't leak pertinent information, proprietary information, customer information, things that are part of its training data. And so this is extremely important, nice thing like we looked at earlier with bedrock, any data that you use to fine tune one of those foundation models, it is not shared with third party Model providers. It's not used to improve the performance of those based models. And so making sure that we're cognizant of that and then some risk here and some of the mitigation. So with great power comes great responsibility, right? With gen AI being new and evolving at a rapid pace, it's important to be aware of these risks and practice responsible AI. So let's look at some examples and some mitigation strategies we might employ. So toxicity, this is kind of harmful inflammatory, offensive content that might be produced. And so different mitigations may be curation of your training data to ensure that those examples aren't there or guardrail models we're seeing being used. So detecting and filtering out some of that unwanted content and hallucinations we probably heard about too. Not all the assertions or claims that these gender of AI models produce are factually correct. And so you wanna ensure that you're teaching users to ensure that they are checking or validating some of these things and cognizant of those risks, intellectual property like we saw so concerns about privacy, making sure that data is secure encryption as well as filtering. You can filter generated content to remove protected content. If it's found remove that match. We're seeing plagiarism cheating a lot in the educational spaces as well. Now there's a lot of benefits to generative AI in education as well, tailored learning assistance and the such. So we need to be able to balance those risks and benefits appropriately and then disruption to the nature of how we do work, this is gonna happen. It doesn't necessarily mean jobs are being eliminated. It's gonna change the way in which task or work is done, but it's gonna also likely create entirely new jobs, some of which we don't even know what they are just yet. And so being uh cognizant of that and talking to your employees about that along the way. And that's a perfect segue here to our last module. I'm gonna go through a little quickly here because I want to make sure we got a couple of minutes for Q and A here at the end. So what is a generative AI organization look like? We're gonna talk about strategies for integrating that into your org. How do you build the people, the process, the culture for success and the importance of having some governance structure to that? And then some actions you can take now right after the session. So starting with preparing your organization, start with your leaders, wanna make sure that you drive leadership alignment and that then trickles down providing resources to your employees for training and other things. So what's your organizational readiness? What's the impact of change to employees and it might change some of your operating models as well? And then moving into your employees start explain what generative AI is. Education is the strongest tool here. There's fear, there's sometimes just, I don't know it's too complex. I don't know where to get started. So it's just large machine learning models like we've seen. And so instead of, um, you know, having those barriers to entry, explain what it is, educate your employees and then address concerns about job security. They might be worried about how things are gonna change or reshape, address those and in fact, foster good communication, change that some of those concerns to excitement. Oh, wait, now, I don't have to do that, that manual tedious task anymore. Awesome. That's gonna free me up to do this and so new roles, new products, new ways in which we might be doing business and then the potential benefits of generative AI to the company as a whole. So automating those tasks, improving customer engagements and interactions, reducing cost, streamlining processes. We saw earlier with that shoe example, generating entirely new content in the form of text and images, um tailored marketing campaigns, you name it, there's tons and tons of possibilities out there. And then fourth, encourage your employees to provide feedback, feedback loops are important with everything we do within machine learning, generative AI and all the such. And so encourage your staff to, to share their thoughts or concerns at work and it could be done, you know, through anonymous surveys, focus groups, one on one meetings, start with your leaders and come up with a plan of how some of those communications are gonna be made. And then based on the input you receive. What's our plan to address this and then have that feedback loop because this isn't just a one time activity, the pace and rate at which gender of AI is evolving, you're gonna wanna be able to continuously capture that feedback. And then also emphasize the importance of continuous learning. If you think you know it all, you likely don't. My favorite thing is realizing that there's something I don't learn, get that excitement about continuous learning because what's out there today might shift in another year, two years, we're seeing rapid pace of innovation. And if we have that encourage that culture within our organization and we foster that feedback and other things you're gonna find oftentimes that maybe a lot of your employees are coming up with new use cases or ideas for you. Once they have an understanding of what generative AI is and the benefits that it can provide a lot of times they know the very detailed level details about specific processes or tasks that your organization or business might do. So the importance of continuous learning. And this is also gonna foster maybe more use cases coming down your pipeline. So how do we organize for success? It doesn't happen overnight. We want to position those teams for success. And we talked about education, continuous learning, communications and collaboration. You might not all necessarily have all the experts required for some of these projects out of the box. Or right on day one. And so collaborate with experts in the field. They can provide, you know, detailed guidance, unique perspectives and advice. And I encourage you. We got a ton of experts here at Reinvent and they're gonna be speaking to some of these topics. So check those out and hang out and talk to me a little bit after the session as well and then focus on data quality and availability. You gotta have good data with anything you're doing. It's essential to producing accurate and reliable systems and then a governance Model, you need that governance Model. And this isn't something you think about when you deploy it. This is something we think about upfront during that scope, a framework for managing the risk and benefits of this new technology. What's the responsibility and accountability of the system? You know, how do we track that over time? Are we sure that we've made ethical responsible AI considerations with fairness and transparency and then closing it out taking some action now? So hopefully, I know this was AAA lot to pack in to our one hour session but recap and infuse that gender divide thinking start with the use case, start with a couple of use cases. Build on that success. Educate your stakeholders experiment too small scale projects, test how it works, build that knowledge and come up with a strategy implement monitor extremely important because implementing it and if you don't have monitoring, you're not gonna have any idea if you're actually delivering business value. So how do you determine that success criteria and highlight your successes and your failures? Don't just brush those under the rub, learn from them. And so closing it out, we got a ton of great resources. Visit us at the training and certification lounge or uh the challenge lounge in the expo. We got tons of team members you can speak with um access to a lot of good material. And, uh, we've got self-paced labs. I alluded to earlier. We got a code whisper, one help, handful of other gen AI ones. Check those out if you're looking to get more hands on and if you don't have a skill builder account, your employees don't have a skill builder account if you take one thing away. Great one. We've got a free seven day trial. You can redeem uh, for some of our subscription based content. There's also tons of free digital content out there. Take advantage of that, hopefully enjoyed the session. We got just a couple of minutes left. So I'll open up the floor for some Q and A and I can also hang out outside the room, uh, when the next session starts.

Select your cookie preferences

AWS re:Invent 2023: Generative AI Essentials for Decision-Makers - Use Cases, AWS Services, and Implementation Strategies

Amazon Web Services

Up Next

Revolutionizing Business Intelligence: Generative AI Features in Amazon QuickSight

Accelerate ML Model Delivery: Implementing End-to-End MLOps Solutions with Amazon SageMaker

Streamlining Patch Management: AWS Systems Manager's Comprehensive Solution for Multi-Account and Multi-Region Patching Operations

Deploying ASP.NET Core 6 Applications on AWS Elastic Beanstalk Linux: A Step-by-Step Guide for .NET Developers

Simplifying Application Authorization: Amazon Verified Permissions at AWS re:Invent 2023