How to work Introducing spring AI

Introducing spring AI

Hi folks unless you’ve been living under a rock or traveling off planet or something like that for the last year you’re probably aware that AI or more specifically generative AI has become quite the buzz recently things like chat GPT and Google bard have brought generative AI to the masses and it seems that everyone is dreaming and scheming of how they can leverage AI in their projects and and even in their day-to-day life if you’re a spring developer you might be wondering how you can Implement
generative AI in your spring applications if so then this video is for you I’m going to introduce you to Spring AI a relatively new project that enables generative AI in Spring boot it’s so new in fact that I’ll be working with a zero zero two Snapshot in this video it’s very much still pre-beta uh so it means there’s a lot of there may be some rough edges but it works really well for what it does right now it actually works very well um in fact it works with openai and Azure open aai at the moment but this
will likely uh change and it’ll likely interact with other generative AI apis in the future and suffice it to say it’s kind of Awesome so I’m not going to make you wait any longer let’s go ahead and get started all right to get started with spring AI I’m going to start by creating a brand new project here in IntelliJ we’re going to call it um songs API all right so I’m going to create a songs API I am going to use Maven no particular reason other than I just feel comfortable using Maven more when people
are watching me even if it’s a pre-recorded video I am going to change the group to hanuman but I’ll leave everything else alone everything else seems fine I’m going to add just a simple uh spring web dependency to my build I am also going to add devtools to my build but there is no check box yet for spring AI spring AI is still new so new it doesn’t show up in start. Spring.
io so just add those two dependencies and we’ll call it good should be good enough we’ll get the project started I’m going to make this window a little bit bigger so we can see everything I won’t have to scroll around as much I’m going to close help I’m going to make one quick adjustment to my compiler so that it will build automatically when I make changes which means that retools is going to be able to um it’s under compiler actually devtools is going to be able to automatically restart my application when I make any
changes it’s going to just save us a little trouble of having to go up here and click a little button all right with that said we’re ready to roll um now I said that spring AI is so new it does not yet have a checkbox and start.spring.io so we’re gonna have to add it but it’s still a snapshot build that we’ll be using so the first thing we can do before we add it is we need to add our snapshot repositories so snapshot repositories um let’s see repository cool and then we’re going to
have ID spring snapshot it looks like a copilot was suggesting something so uh maybe I should have kept it we’ll just see we’ll go ahead and keep going until it tells me that it’s something I don’t like but that looks good so far and uh do we want snapshots enabled absolutely we do and that’s good that’s what we’ll that’s what we use that actually I think is what I need all right with that said again sometimes I rely on on co-pilot to do things and I trust it and then I find out later on it did something that had I
been paying a closer bit of attention I would have not done but I think that’s right never going to add our dependency it’s going to say dependency and the dependencies specifically that we’re going to add is spring a AI and we have two choices here we can either use just good old-fashioned open AI or we can use Azure open AI but I’m just going to use good old-fashioned open AI spring boot starter and that’s going to suggest.

Introduction to Spring AI Online Class | LinkedIn Learning …

ai which is what I want and it’s going to also suggest a version I’m going to add 0.2.0 snapshot very good all right with that said we have all the pieces in place dependency wise so we’re going to actually start start working with it now oh actually there is one other thing we have to do there is one other small kind of important thing we have to do you need to go set your open AI API key somewhere now I’m using open Ai and I am going to use an AI key an open AI key that uh is mine I don’t want you to use it
but I’ll show you where to set it if you come over here and go to Source main resources go to application properties you can say spring.ai.openai dot API key equal and then whatever your API key is go you’re going to need to go to open AI register get yourself an API key and use it now I’m not going to show you typing my open AI API key here instead I’m going to go set it as an environment variable which is going to let me do this I’m going to go go pull in open AI API key that way I’m not hard
coding it in any of the code that I might check in to GitHub not that I’m necessarily going to check this into GitHub but if I did I wouldn’t be hard coding my openai API key and then checking it into into a public GitHub repository so I’m just going to reference this environment variable and I will set that environment variable separately all right so with that said I’m going to go ahead and close this and now we’re ready to start writing some code so I’m going to do a really simple thing I’m going to compare to
Java and I’m going to go to the Java folder that is I’m going to go ahead and add a new class A Java class and we’re going to call it songs controller it’s going to be a class just like any other rest controller would be I’m going to annotate it with rest controller I’m going to put a request mapping of Slash songs on it that’s going to be the base path for all the requests that this is going to handle and then I’m going to inject into it by via its Constructor I’m going to
inject into it not a song repository uh clearly copilot has no idea what I’m about to do I’m going to inject an AI client so AI oops AI client I’m going to assign that to a instance variable I’m going to go ahead and go and create that field cool we have that there now and we’re ready to start using it let’s see I’m going to handle a simple get request so get mapping four slash top song not a very useful example that I’m creating to start off here but it’s going to give us something to start to
kind of work with I’m going to say string I’m going to call it the top song method and I’m going to ask open AI this question I’m going to say what was the billboard number one year and top 100 single for 1980 you may want to guess what that is maybe you know already don’t Google it that’s cheating we’re gonna find out in a minute anyway we’re going to ask open AI to tell us that and I’m going to return the answer aiclient dot generate that’s the method we’re going to pass in
because ultimately what generation means is you can kind of think of how this AI stuff works this generative AI stuff works by thinking about it in terms of something that you’ve probably seen before on your phone when you’re typing a text to someone or typing something and it’s offering you a recommendation of what the next thing is you’re going to type the the basically Auto completion the suggestions it’s going to offer those things to you and it’s but it’s a little bit kind of written large
if you will rather than suggesting what a word should be based on the context of what you’ve typed already this is going to suggest the next thing that it thinks should follow after whatever the prompt is now in this case The Prompt is a question but the prompt could also be you know a command such as create something or or or produce something or or imagine something and then what this is going to do is based on that context based on that prompt it is going to generate what it consists considers a completion for that prompt
so in this case The Prompt is a question and so it considers the completion and answer to that question so what was the billboard number one year in top 100 single for 1980 to complete that prompt we’re going to ask it to generate an answer we’re going to say generate given that prompt and it’s going to generate an answer so let’s go ahead and kick this off let’s go ahead and fire it up I’m going to pull over a window to poke at this with I’m going to make the font a little bit bigger and we’re going to use HTTP IE if
you’re not familiar with what HTTP is it’s kind of like curl but a little bit easier to work with among other things it defaults to localhost so I don’t have to specify localhost I can also do other cool things with it it assumes Json output by default rather than um anything else and likewise for input so it’s just easy to work with but I’m going to say songs slash top song I think that’s what my path was let’s double check before I hit enter and yes it looks like that’s correct
and it’s going to say wait for it the billboard number one you’re in top 100 single for 1980 was Call Me by Blondie everybody likes that song apparently in 1980 they did uh so there we go it worked that’s the simplest thing we can possibly do to add generative AI to our spring application is just simply give it a prompt and have it do a generation against that prompt now another thing that you can do is we don’t necessarily have to give it a string prompt directly we can also give it a richer object called an AI prompt
but we’re not going to do that in this video that’s a more advanced topic the key benefit of doing that way is you also get back a response as an object not as just another string what you’ll get back is something called you give it a prompt object and you get back in AI response object as your output and the AI responds out object has with it some additional metadata some metrics around the usage and this is a good a good thing to know because ultimately this is not free when you’re using an open AI
it’s not free it’s going to charge you for use based on how many tokens you use if I were sending it a prompt and getting back an AI response the AI response would have some additional information in there telling me how many tokens I’m using but I’m going to talk about that in this video we’re just going to move on and instead our very next thing we’re going to talk about is how to templatize this because right now among other things I have this hard-coded as 1980 what if I wanted to
know what the top song was for a given year maybe 1981 or 1975 or what you know some other year Well to do that we’re going to make some subtle changes to this I’m going to put in first off a path variable into the request so top song for some year I’m going to take as a parameter a int that’s referencing that path variable for year and now I’m going to change this prompt a little bit I’m going to put on another line so I don’t have to scroll back and forth quite as much to see it
and I’m just going to go plug in instead of the 1980 I’m going to plug in just a placeholder for a year now at this point I need to somehow get that year plugged into that string because right now that string just has curly brace here and it doesn’t have any year that’s passed in from the from the API request so what I’m going to do is I’m going to create a prompt template there we go prompt template I’m going to pass in that prompt and from that prompt template I’m going to say okay template
I’m going to say not not with I’m going to say add year and I’m basically giving it a value for what it’s going to plug into there in the end that that whatever is passed in as the request the path variable on the request that is going to be plugged into into this model and that’s going to be plugged into this before we get a chance to ask the um the llm our question for generation so you notice I’m passing in a single value here this is the easy way to do it you can also add a map so if you have
multiple values you can either call template.add several times or you can pass it a map with all your values in it whichever one is best is up to you but for now we need to new we do need to do one more thing because at this point prompt is still this string it doesn’t have anything plugged into it yet so I’m going to say okay template dot render very good and that’s it we’re just we are now using a template to fill in our prompt and now that devtools has restarted it we’re ready to kick the tires on it
again now this time I can’t simply say top song I have to give it a year so how about we go to the very next year 1981. and we’ll find out that the billboard number one year-end top 100 single for 1981 was Betty Davis Eyes by Kim Carnes I got to be honest not one of my favorite songs ah but oh well I mean to each of the round every if you like the song good for you it’s not one of my favorites but all right that was the number one you’re in uh one top 100 single for 1981. let’s try another year
how about we try 1989. and we’re gonna get that it was Look Away by Chicago we could ask one more time just for grins and we could say maybe what was it for 2020 and we’re gonna find out I already know the answer to this we’re going to find out it was blinding Lights by the weekend so there we go we have now an API it’s going to tell us what the number one was for any given year well that’s great except that it’s giving it to us as a string maybe we don’t want that as a string maybe we
want it as an object some some domain object so let’s let’s see how to do that to do that we’re going to create a new well Java class no we’re going to call this top song but not a class we’re going to do a record I mean a class would also work but might as well do a record I mean we are talking about songs and albums are often called records I mean yeah never mind that all right so we’re gonna have the songs title we’re gonna have the artist we’re gonna have what album it was on and we’re going to have
whatever the year was okay those are the four things we’re going to have in our top song record great now let’s go back over to our controller and we’re going to make a few changes here first off um I’m going to start off by creating what’s called a output parser there’s a handful of output parsers that’s bringing AI comes with there are some for dealing with lists some for dealing with maps I’m going to be dealing with just a simple bean output parser and what Bean output parsers do is they take
and parse the what whatever comes back from the AI request from the generation it’s going to parse that into fields that can in properties that can go into a bean or in this case a record so I’m going to say parser equal new being output parser I’m going to use top song now here’s the thing though you notice that when I ask it about a question about a top song it came back as a sentence how is it going to map that sentence an English sentence into properties in a bean well that’s kind of a problem isn’t it
we need to tell it what format we expected to come back in and it is possible to add into our prompt some details and say please bring this back as Json output here’s the fields you should have on it and all that stuff there’s a lot of stuff you have to do to tell it that and I don’t necessarily want to do that all the time instead what I’m going to do is I’m going to change this name here to uh prompt string just because I might reuse the word prompt a little bit later on and I’m going to add I’m going to also
change it to a multi-line string because I just I just want to say a lot I have a lot to say in this this example so I’m going to copy this out of here get rid of the old double quotes put that in there and then on another line I’m just going to put in there the word format as a as another placeholder is another part of my template placeholder I’m going to put in the word format so now we’re still asking the same question what was the billboard number one year in top 100 single four some given year but we’re also passing
in whatever its format is now as far as a part of our prompt template I did change this from prompt to prompt string so I need to do the same thing down there I did add the year that’s good I’m going to say also add in something for format now how do I know what the format is well I could certainly dream up as part of this and in these quotes say give me Json output uh here are the fields um blah blah blah I could do that but that suddenly gets out of hand that gets ugly so what I’m going to do instead is
I’m going to do something very clever that’s that spring AI does I’m going to ask the parser hey what output do you think there should be and I’m going to say get give me the format string and what’s great about this is the bean output parser knows exactly what to say to tell open AI to give me this in a format that it can later use to parse the response into a top song that’s awesome it’s already built in ready to roll very good so now one other little thing is I need to say okay
uh template you are going to use the output parser called parser and what that’s going to enable it to do is do all this magic that we’re looking for here is it’s going to give me that format it’s going to give it back to me in Json not as a not as just string and I’m going to say okay prompt prompt this is where I’m going to have the new prompt variable is going to be template dot render or I’m sorry template.
create that’s not what I meant to do template dot create there we go and then I’m going to say okay uh AI client I want you to generate from that prompt I’m going to assign that to the AI response now this is the object I talked about earlier where the AI response could contain additional information in it but I’m just going to keep it as simple as that in fact I’m looking at this now I’m not even sure why I even bothered with a response although I’m sure it’s a fine Choice here um I could have just passed in a string
here and got a string back but that’s not really what I wanted to do um and I’m going to say okay I’m gonna get the text out of it so yeah we are getting taxed that’s for sure I’m gonna getting AI response dot get text that’s going to still I’m sorry I did that wrong I forgot to call generate so get generation dot get text there we go and now that we have that text what that what what’s contained in that text I’m not going to show it to you right now because well I have to run it and put
some debug in there but I will tell you what it is it’s going to have in it essentially Jason assuming that the git format that I gave in as the format to The Prompt is assuming that does its job then this text this string here is going to just have a whole bunch of Json in it so I’m going to say hey parser um return instead of returning all that I’m going to say parser.

parse oops helps if I can type right parse text and I don’t need all that other stuff I just simply needed to give me back whatever that is great so what did I do wrong I got an error here oh I’m returning a string yeah I’m up here I’m still returning a string I’m not returning a top song so we need to fix that there we go and now everybody’s happy cool so what we’ve done just just a quick recap of all this code is I’ve given it a b now put parser that knows how to deal with song top song I’ve given it the same prompt
string I gave it before with a placeholder for a year but also a placeholder for format which I populate by asking the parser to tell me what format it wants in fact just for grins while we’re here let’s go ahead and just sis out that or I’ll do a CIS error because it stands out in in red when I do that that’s going to have our format string of parser dot get format that way you can kind of see what it what it passes in as that prompt all right we’ll see that in the in the output here in the console but
once we have that then we go ahead and submit our prompt for Generation we get back an answer in text but it’s not necessarily human English text it’s Json text and we just ask our parser to turn it into an actual top song object that we return great let’s give that a shot and see what happens now I don’t have to restart it because devtools has already restarted it let’s come back over here clear the screen so we can see the new stuff apparently it restarted again I don’t know what I did to cause that but okay
uh then we’re going to say okay what was the top song for 2020 and I’ll show you what the output was here in a minute or what the the format was in a minute but as you can see we now got Jason back we got an object of type top song that’s giving us blinding Lights by the weekend it came from the album blinding lights Year 2020.
great let’s try that again for 1981. we already seen the answer for 1981. we already know what the answer is uh we’re gonna see that it was Betty Davis Eyes by Kim Carnes but let’s see it in Json format oh well that was odd it gave us physical by Olivia Newton-John now here’s an example even though I before creating this video I poked at this a dozens of times and it gave back frequently gave back uh Betty Davis eyes which is in fact the correct answer this time it gave us physical by Olivia Newton-John which is kind of peculiar to
me that it gave me a different answer chances are good that my my wording in my prompt my my choice of words here left a little bit of ambiguity so if I was a little bit more clear in my prompt it might make it more clear that no I want that one but regardless we can try it again just if we get Betty Davis Eyes by the way I’m not a huge fan of the song physical but I do like it slightly better than Betty Davis eyes there we go it doesn’t have an album name for some some reason I don’t know it just didn’t
know it let’s ask again for 1980 we know the answer for 1980. we’ve seen it before I am hoping sincerely hoping it’ll be consistent with what we saw before it’s Call Me by Blondie good job he still didn’t know the album name that’s peculiar to me but that’s okay uh we won’t worry about it right now the fact that I got an object back with these these answers in it is awesome for me let’s try one more we did 1989 I don’t remember what we determined the answer was for that
but we’ll find out again Faith by George Michael I’m pretty sure that was I’m sorry no Look Away by George Michael I’m pretty sure that was not the answer we got before but again I probably need to do something with my prompt to make it a little bit more um specific and a little less than um ambiguous on what it’s returning so I don’t know maybe that’ll improve it uh speaking of which there is an entire um discipline forming around this for prompt engineering where defining the prompt
clearly is is actually a skill to have clearly and with the fewest amount of tokens um and act so it gives an accurate and consistent answer uh clearly not something I’ve employed in this example but it could be done now about that format what is it what does that format look like so let’s see there’s got a few of them here for this last one we we have your response should be in Json format do not include any explanations in other words don’t say here is your Jason followed by some Jason don’t include explanations
only provide RFC 8259 compliant Json response following this format without deviation in other words don’t go off and do your own thing here give me some Jason back and by the way here’s the Jason schema that your output must adhere to uh album artist title Year all of type string great and to determine that based on the fields that top song provides so that it could later take that Json that he gets back and bind it into that so great now we’ve seen how this works let’s do a little bit of recap
in this video we hit the tires on Spring AI submitting simple prompts templated prompts and then we broke away from the string responses entirely by using an output parser to both alter The Prompt with schema information and bind the response to a domain class but there’s a lot more that spring I can and will be able to do that we didn’t cover including retrieval augmented generation or rag for short provides additional information in the form of documents to the llm that the llm may not have been trained on
because those documents may be large and therefore involve lots of tokens which means they’re going to cost you more you could also use with them embeddings and Vector stores basically to pre-select relevant document Snippets essentially the documents are converted into Vector data and stored in a vector database then instead of extending all of your documents in a prompt the vector databases search for similar and therefore likely relevant document Snippets this keeps the the size of your prompt smaller and therefore keeps your
costs smaller as well we also didn’t talk about functions and that’s because at the moment spring AI doesn’t support functions but it likely will in the future these enable an llm to indirectly interact with one or more functions that you provide that your application provides these functions can provide additional structured data that can be used in generating responses or even functions that can trigger some Behavior such as maybe ordering a pizza or turning on a light or something like that we didn’t talk about chains chains are a
way to string together several prompts taking the results of one prompt and feeding it into the next prompt and and that works great it’s it’s fairly predefined what these chains are you know what step one is step two s step three is fairly rigidly structured but very valuable way to produce richer answers from a series of questions that are being asked now if you want to do something a little bit more fancy there’s also the idea of Agents agents offer functionalities similar to chains but instead of
strictly defining a set of steps they allow for decisions to be made as to what the next step should be well these are all great topics and things that spring AI either already supports or will likely support in the future and it’d make a great future video on more advanced spring AI topics but for now thank you very much for tuning in I hope you enjoyed the video I hope you learned a little something from it here’s where you can follow me on all the social medias here’s where my books are but if you don’t remember anything else
remember to go to www.habooma.com there you’ll find information about my books other videos that I produced all my social media contact information and things like.