Don't Trust, Verify Heavily

00:00:10: Hello and welcome to the Cygnus podcast.

00:00:12: My name is Michael Doudin, joining me today is Venkat Subramaniam an author speaker and problem solver.

00:00:19: Venkat thank you so much for joining me Today.

00:00:21: Hey Michael Thank You So Much For Having Me.

00:00:23: it's a great pleasure To be here on The Show!

00:00:25: One of things we've been talking about A lot lately Is how do use AI to support development?

00:00:30: And specifically How Do Use It To Solve Some Problems In Code Such As Identifying and Fixing Issues?

00:00:37: What Can You Tell Me About This?

00:00:40: The realization for me is AI, it's not artificial intelligence.

00:00:45: AI to me stands for Accelerated Inference.

00:00:49: It's an amazingly fast inference engine.

00:00:53: Now let's take that idea For a minute and relate That.

00:00:57: do the consequence of it?

00:00:59: The problem with inferences and inferences neither right nor wrong And inference Is a reflection Of their data provided To it.

00:01:07: you know without respect.

00:01:09: You know, I'm a programmer but i don't consider myself to be writing super good code.

00:01:14: Uh...I write terrible code!

00:01:18: But my superpower is in finding fault with other people's codes and as it turns out AI has been trained on a vast amount of information and two specifically.

00:01:32: One is all the code humans have ever written, some of them really good, a lot of them are bad.

00:01:38: so as result unfortunately when you ask AI to generate code that quality's reflective off what it learned from but other set up input which very significant is blog post tutorials books articles, course materials.

00:01:57: and what is the distinction between these two?

00:01:59: A code reflects a software system but not great quality.

00:02:05: But book or tutorial for example right if you were reading on functional programming are reading of book on concurrency.

00:02:15: What do books tell us about the do's & don'ts?

00:02:19: so as it turns out AI can infer from both set of information, but in a separate way.

00:02:27: So when you point the piece of code to AI and ask it to identify errors in it?

00:02:34: It is super good at identifying errors!

00:02:37: You know here's something I've done for past about an year... I have this privilege and luxury-I'm very humbled by that-to be in front literally thousands of programmers, you know every year and I take a set of problems And i've literally shown this to about a thousand five hundred programmers.

00:03:00: And off the thousand five-hundred programmers for one problem especially About three out of the thousand Five hundred have kind of come close To knowing what?

00:03:12: The problem is but they still didn't hit it.

00:03:15: But every single time And using multiple AI models, when I feed the same exact problem into it.

00:03:25: It may sit down which is what fascinates me.

00:03:28: so my analysis today as while AI can be used to generate code its superpower is in really finding issues with software systems.

00:03:40: Now we have already seen evidence of this and other cases as well, but this something I've noticed.

00:03:46: fundamental code you know has a programmer.

00:03:49: either i have stubbed my toes over time or I don't know which is more painful to watch me stubbing or my clients doing that.

00:03:58: But when the clients have sent me sample code and saying, here's a quote i'm stuck with this issue we're having in production what am I doing wrong?

00:04:06: So I took those code examples modified it too protect their client information but fed them into AI on every single time It nails.

00:04:18: so I think The ability of AI to infer plays a vital role to our advantage when we use AI to detect issues and identify problems.

00:04:31: And the other problem with us humans is, We are really creative but we're not good in dealing with cognitive load!

00:04:40: I often say...we like the salt that's nailed as it comes to cognitive load You throw complexity at me and i wither immediately right?

00:04:50: And the interesting part about AI is, what is so hard for us?

00:04:54: AI deals with that effortlessly.

00:04:57: It doesn't have cognitive load!

00:04:59: So when I take a seven thousand lines of code or twenty-thousand lines which unfortunately as a client...as consultant..I'll have to deal from clients side and I throw this to AI, it turns around and performs the analysis.

00:05:15: And provides a reasoning as to what the code is doing?

00:05:19: Now not that i'm going to trust everything its says but gives me the right direction for forward an analyze further.

00:05:27: so overall my observation while AI extremely powerful very non-deterministic super power for us developers identifying issues in code, dealing with production issues and to be able to reason about the code and deal with cognitive load as well.

00:05:46: Yeah which is just fantastic.

00:05:50: I know one of the things that you and i have both talked about over the last few years how the way we name things impacts our ability to read it.

00:06:07: The importance of readability code, and I think that's probably even more true today—we read code a lot more than what you write in today.

00:06:16: How would say good clean readable code?

00:06:21: What is important for that today versus five years ago?

00:06:26: And why do they need technology to maintain this over time?

00:06:32: The quality of code is all the more important today, maybe ever so than it's been in the past.

00:06:38: And then to the point you just made for example that we're going to be reading code quite a bit.

00:06:42: but the issue is this You can outsource your coding to AI But you cannot delegate Your responsibility and reputation.

00:06:55: This kind of goes back to something that reminds me off, twenty years ago.

00:07:00: As a consultant I had developers working on my team but i was the client facing and when it went into my clients site My bosses at the client's site would ask questions or explain how certain things work Or asked me make quick change.

00:07:21: And I didn't have the luxury to throw my hands up and say, well it doesn't matter who physically wrote this.

00:07:37: In that past, twenty years ago it was a fellow human writing code but I were still responsible.

00:07:46: So i would sit there and read the code understand that.

00:07:49: And so when they went to my client's site ,I was able to answer them.

00:07:53: Today you have a much faster machine That can write lot more code and scary quality code if you're not careful about it.

00:08:03: So this really comes back to our responsibility, we still are responsible for what.

00:08:15: we can just throw our hands up and say, hey I did it.

00:08:19: So that really brings back to a few things right?

00:08:23: It's the maintainability of code.

00:08:25: but... And this is where i'll be honest-I'm still debating in my head about it there are certain things that are much more important from human maintaining code which might get less relevant over time when AI is going to maintain their code.

00:08:44: But then there are things that are extremely critical.

00:08:47: Things like modularity, things like cohesion, things low coupling... I'll give you an example of this!

00:08:55: I asked AI to generate an early application doing spectrum and development.

00:09:02: as a experienced programmer The difference between a novice and experienced programmer is, A novice thinks about what would work.

00:09:10: And an expert things about what might not work right?

00:09:14: So the unit I asked AI to generate this application.

00:09:18: On my mind, it's not the question of what is going to work but actually fail.

00:09:24: The minute it launches the application I turned off the Wi-Fi and disconnected internet connection then said go do this task.

00:09:34: And it failed miserably because he didn't know how to handle that situation.

00:09:40: Now in fairness, may I say gosh you don't tell what to do?

00:09:43: Well but reality as an experienced developer when your developing application the first question we're going ask is hey this what is the behavior we expect me to do, or maybe we can even reason that there's a logical conclusion in terms of how we need it handle which isn't what AI had done.

00:10:10: So if I don't think through it's going to become problematic.

00:10:18: So what I did is immediately went into the code generated by AI, and i'm looking too see What It Is Doing?

00:10:25: And now I am modifying this specification to say hey when This situation arises you need To make this change.

00:10:33: but The question is When You Look At The Diff Of The Code Change If The Code Had Been Cohesive if It Was Properly De-Coupled And Loosely Coupled Then The diff Is Gonna Show Minimum Changes With A Code which means it becomes easier for me to verify the change.

00:10:50: that's happened as well.

00:10:52: But on the other hand, if this was a ball of mud where there is no cohesion right?

00:10:57: So without guiding AI properly It decided to mix UI and logic together.

00:11:03: We all know not to do with.

00:11:05: we have done it but when you see I do what you shake your head and say No, we don't do that.

00:11:16: When we were writing code, We had to coach fellow developers and the effort was manual.

00:11:23: Right?

00:11:24: But now...we still need to coach but what you're coaching is that tool rather than a human being And because The effort is no longer manual.

00:11:36: I think this Is me right visual thinking.

00:11:41: there are so many disciplinary practices, I call them practices that lead to sustainable agile development.

00:11:51: That we only talked about but could never do because of so many constraints.

00:11:57: maybe with AI it opens the door for us.

00:12:01: finally bring some discipline create sustainable software and my hope is there are tools that we can use if only will have a little bit more discipline, not to rush with great speed now.

00:12:21: That we have a tool just slow down a little bit so we can leverage the speed, but at the same time deliver high quality as well.

00:12:30: So I think that quality is playing more vital role today than ever before especially in it's kind of like you know The damage You Can cost by riding a bicycle versus driving A car or flying right?

00:12:47: That Is the Comparison i want to make here!

00:12:50: As the wonderful saying goes with great power comes greater responsibility.

00:12:55: We got a very powerful tool, but we need more responsibilities so that you don't end up in a wreck.

00:13:02: No I absolutely agree and it's interesting with the concept of dealing with another human.

00:13:13: If I'm doing a code review for an other person and i know they put a lot of work into it, the empathy in me is going to say uh...I don't want you'd have to rework

00:13:21: this.".

00:13:21: You know?

00:13:21: This is okay now but maybe next time Don't-.

00:13:24: I don't care if I had make AI rewrite at twenty times right.

00:13:27: that's not a problem

00:13:28: The point!

00:13:30: Totally true.

00:13:32: And one of the realizations For Me Is Uh.. I said these days when you see AI do terrible job Don't complain about it, because there's no point in complaining.

00:13:46: Instead go after improving your skill files.

00:13:50: Go After Improving Your Input You Provide To AI.

00:13:55: So to me, improving what AI provides ease an error rate of an incremental process in itself.

00:14:03: We're going to discover flaws, we're gonna discover areas but there is also unfortunate consequence.

00:14:10: you might provide a very solid set off skill files and context files But the problem at the fundamental level AI is nondeterministic and inconsistent.

00:14:23: So just because you said follow these things, it doesn't mean it actually did.

00:14:30: so You may have to really go back in review with right?

00:14:34: But this where the lessons I'm kind of taking back as my context files and skill files also are cohesive Does that's more focused they are The more effective they are.

00:14:50: AI tends to ignore parts of it because its context really grows too big for it to maintain in a sensible way.

00:14:57: So when you look at these things, this is an iterative process You know, I often say people come to me and say how do we make AI deterministic?

00:15:11: And they tell them oh it's very easy.

00:15:13: As soon as you have straightened the dog's tail that is a next task which can go after.

00:15:19: so the point really fight against the nature of something, right?

00:15:27: The AI is non-deterministic.

00:15:30: So the question is how can we alleviate the nondeterminism rather than eliminate it?

00:15:35: and I think that's where lessons are taking back as if you're more cohesive or much more focused on targetting a clearer by way of giving examples.

00:15:50: It's super good, rather than just saying don't do this or do this instead to show an example code and say here is the reflection on what it may look like.

00:16:03: So things like that, I think we can take back as developers and learn.

00:16:08: And to your point the tools don't get tired.

00:16:12: We get tired!

00:16:13: Right?

00:16:14: And so we can learn from that experience and say am i improving the behavior of the tool over time?

00:16:21: but here are the lessons.

00:16:24: To

00:16:33: touch back on something else you said, to extrapolate a little bit.

00:16:37: When a junior developer sees the specification for software-to-write they're going tend take it at face value but maybe automatically like fill in gaps if there's a constraint missing or, or a condition missing.

00:16:54: They're going to just naturally fill that in because that's the way code has to be written and not necessarily think about stopping it saying hey how should I handle this?

00:17:04: A more experienced developer is generally gonna say Hey you need to tell me Because i know there are multiple ways they can handle this but at the same time their gonna feel different gaps Their gonna feel like You didn't specify that this needs to be secure.

00:17:20: You didn't specify how it's going to get deployed, you didn't specified all these other things they understand technically.

00:17:27: I need to do this and bring the table as my expertise then mesh with your specifications.

00:17:34: How does AI fit into a mold of how we communicate what we need build?

00:17:43: What are roles involved in making communication work?

00:17:47: Yeah, that's a really good question and here is the scary thought about it.

00:17:52: You know we talked these two extremes which has been our reality right?

00:17:56: We got a junior developer who was in experience but there's an urge to fill-in-the-gap And senior developer knows things and knows when not.

00:18:15: So if you put those two in these two extremes, I wish I could say AI is the middle.

00:18:22: But AI isn't some faraway extreme on its own.

00:18:27: think of AI as this maniac who has the ability to plagiarize with no guilt whatsoever?

00:18:36: Well This goes back to my statement that AI Is not about intelligence but about inference.

00:18:42: so The sad part that when AI sees the gap, it sadly fills it in with things it infers.

00:18:51: And so the question it runs across is where have I seen this before and what have i seen most common?

00:19:01: So unfortunately, AI looks for.

00:19:06: It doesn't look out for what is right.

00:19:09: For you your product, You are being.

00:19:12: so what it may fill in might not be appropriate far your application.

00:19:19: It might fill in what it has seen, and that doesn't even mean if the right thing is seen maybe its most common or a random thing it's seen and pulls into this.

00:19:30: And when I look at code being generated there are times my mind literally asks me where did you drag?

00:19:44: That to me is a very sad part in my mind, because I respect the value of developers who put effort into creating things.

00:19:56: And if somebody were to steal things that I did without permission... I'm not going to be entertaining that.

00:20:04: No, here is a situation where AI literally drags things in.

00:20:09: this happened about a month ago and it was a little bit shaken up when they saw there's.

00:20:13: It doesn't happen too often but if you're curious I think this code and I am like, gosh.

00:20:33: And I take the repository go down and look at it and have to admit It's a person that don't know and uh...it is a code That is similar To The Problem i was solving.

00:20:47: No!

00:20:47: I Don't Think I want to be using A Similar Code To That quality, right?

00:20:52: I'm not judging at this point.

00:20:54: But to me it's like infringing on somebody's you know intellectual work.

00:20:59: and i think that is where the danger is-is That we looked these two extremes.

00:21:04: but thats a third extreme.

00:21:06: Where It Is Not Looking Out For Us It Is Looking At Stuff Others Have Created.

00:21:14: And if This Were You Know An Education Setting I'll Be In Trouble for Plagiarism you know, as an instructor.

00:21:23: I've been a instructor for awhile teaching part time and this is where over the thirty years, thirty-five years The students who have gotten into trouble are not the ones Who didn't know that stuff because That's what we're there to teach them It.

00:21:37: so once they got in trouble or the one that stole somebody work And claimed it their own?

00:21:42: Yeah!

00:21:43: And i think that Is the catch here.

00:21:45: So... This kind of goes back To if I'm a novice using AI And if it fills these gaps, do I have the ability as a beginner to question and challenge?

00:21:57: And say gosh why did you really fill this in instead of really considering other different alternatives.

00:22:05: Whereas an experience programmer You probably go look out and say hey This is not answered here Are doing right thing.

00:22:14: Right Again comes back for us as developers to use the tool.

00:22:22: But I think we both, right?

00:22:24: The two groups react to it very differently one more definitely versus the other maybe but naively not thinking that this may not be in.

00:22:41: We started this conversation by talking about how you're using AI to identify and fix issues in the code, but during that conversation You brought up the perfectly valid point.

00:22:51: The AI on its own doesn't produce good code.

00:22:55: so How do you kind of make these two?

00:23:00: Opposing viewpoints work together like what is your process for them?

00:23:04: Totally, right?

00:23:05: And that is the surprising aspect.

00:23:08: Is that duality of AI maybe, so it is at the same time a tool or weapon depending on how you look at it which is scary part but to me the key is to leverage where it's really good and figure out ways to improve with, but they say that first step in solving a problem.

00:23:42: One of two things has kind evolved into my mind as something I'm beginning to see quite a bit and the dumb becomes dumber in the age of AI.

00:24:00: And so what that means is, AI is an amplifier?

00:24:04: So to go to your question surprisingly I would say The first step in improving What AI does Is To improve the user Of the AI.

00:24:15: I had somebody come to me and Say if i don't know the fundamentals AI will cover my back.

00:24:22: and I said with respect If you don't know the fundamentals, AI will expose your back in public and the most humiliating way.

00:24:36: somebody random came to me and said, I heard you're a programmer.

00:24:39: Are you worried that AI is going to take away your job?

00:24:42: And I'm like gosh how do we respond with this danger?

00:24:46: who pops this

00:24:47: question?".

00:24:48: Sometimes these things come into your mind instantaneously but they come out of the subconscious mind i think When I ask AI to do something, have no clue about.

00:25:19: Don't do that and change the course correction becomes immediate.

00:25:36: Or, hey you're heading in the right direction but here are other things we need to really improve and focus on.

00:25:43: So so for me a call-to action if you may is If You Are an Organization Invested In Using AI invest in your developers, so that they continue to really improve their fundamentals.

00:26:01: So that they can leverage the AI properly and don't lead you down the path of no return where it becomes extensive to fix it?

00:26:09: That's my first thing about what you can do to improve.

00:26:12: The second thing right this is about improving Shifting from that, right?

00:26:18: The smart becomes smarter the dumb become dumber.

00:26:21: We need to really invest in their developers and continue learning.

00:26:26: but a key here is critical thinking.

00:26:28: if I don't have enough ability to critically analyze ,I'm in trouble shifting a little bit further.

00:26:37: .The second thing to improve as another moment in my mind Trust but verify.

00:26:48: The new

00:26:48: norm, don't trust, verify the heck out of

00:26:53: it.".

00:26:56: So to me this other aspect is we have been screaming through our lungs as an industry for the past thirty years and there's so much that has done in CICD over the years tools to verify the quality of code.

00:27:15: We have tools to identify certain things in software that are indicative of maintainability versus horrible to deal with in production, right?

00:27:29: One of the things to me, engineering in software development is about that rigor.

00:27:35: So one other thing I often ask my developers as when you're developing something simply pause and asked a question what's the cost off failure?

00:27:46: If i can answer you the cost or failure You can tell me hey Venkat your gonna work on this piece of code.

00:27:53: if these were to fail What's the consequence?

00:27:56: I'm going to say, you know if this fails a user is grumpy and they're gonna complain about it.

00:28:04: No big deal versus.

00:28:06: If This Fails It Could Potentially Result In A Loss Of Major Revenue.

00:28:13: Maybe People Cannot Fly For The Rest of Their Day.

00:28:16: Right.

00:28:17: It Can Result in A Few Million Dollars Of Loss maybe Even A Billion Dollar Loss.

00:28:22: my approach to writing code is not going to be the same for two different things.

00:28:27: So, so the consequence of what we do and work backwards from that?

00:28:33: Absolutely!

00:28:35: The question here as sure We are using AI That's great.

00:28:39: but ask a consequence or failure off that code.

00:28:42: And if the consequences higher bring more rigor into what you do which means go back and implement a fairly good CICD pipeline, provide much more verification stages in there.

00:28:58: Sure AI can verify certain things but because AI is inconsistent you cannot assume AI really did the verification or even the verification of a proper extent.

00:29:13: So we still need to have a stage where, We go back and review what's been verified To see if A If we have Useful test right that The problem is not having tests.

00:29:28: You know any any fool can write test.

00:29:31: But the question Is are they test relevant?

00:29:33: Right And I often say, a performance is important.

00:29:37: Don't talk about it right at test for it.

00:29:39: if scalability is important don't talk About it.

00:29:42: write a test for It.

00:29:43: so If you can review those Test cases and ensure Those tests are executing with every change the software Is going through then i think we Can improve The outcome We Are getting from it.

00:29:58: So to me that two aspects To this as A you know, help your developers continue to learn and further their fundamentals improve that critical thinking.

00:30:09: And improve your development process so we can have proper checks in verification and validation.

00:30:17: do take the time to verify that the verifications are actually running properly of course actual good quality.

00:30:28: Now, here's one of the problems that goes back to the fundamental right?

00:30:31: I teach quite a number of courses these days on using AI and i ask ai to write test And critique this test and I usually tell them, give me a thumbs up or a thumbs down on the middle saying you don't have way to evaluate it.

00:30:51: And then a lot of developers will give us thumbs-up.

00:30:53: an few of them we'll get that comes down in and say why are they giving your thumbs down?

00:30:57: They would say well these tests is a poor quality because... Then fill into rest details right!

00:31:05: That's key as.. You're able use AI to write test cases.

00:31:10: I do create these validations, but you need to be able still say is this really of good quality?

00:31:17: This kind goes back capable of writing bad code, I am able to write in that test.

00:31:25: Well we already know AI is capable of righting backcode so it's also capable a writing backtest as well.

00:31:32: So we cannot simply this.

00:31:34: has had discussion yesterday with somebody and they said gosh can you just not use AI to verify things?

00:31:41: And like yes But remember, it's inconsistent.

00:31:45: Remember that if not guaranteed to do the right job

00:31:50: you

00:31:50: still need really be there to validate and verify it.

00:31:55: I think two things.

00:31:57: consider what we get better results out of AI.

00:32:03: It is a tool.

00:32:09: it's an inference assistant, right?

00:32:10: Because that is exactly what its doing under the covers.

00:32:14: It helps I think change.

00:32:17: we maybe have this conversation a little bit in a similar thing to what you're talking about where, You don't necessarily just want to trust the AI and that you need to apply context which is What are the consequences of failure or an error right?

00:32:36: And this-the similar thing applies To the concept.

00:32:39: Where people say well Netflix's doing X Or Facebook was doing Y Or Google Is Doing C I wanna...I need to do That because they're doing it.

00:32:45: Hold on!

00:32:47: What's The cost Of Failure If Facebook Screws Up?

00:32:50: Or Netflix?

00:32:52: Nothing Nothing, it's inconsequential.

00:32:54: Is that money?

00:32:55: Yeah It's money.

00:32:56: What's the cost of failure if your autonomous self-driving agent screws up?

00:33:03: Catastrophic?

00:33:05: you're medical device software catastrophic.

00:33:08: right and We need to look at it from that standpoint as its not just in how we apply AI but how we choose Who do model our businesses after.

00:33:24: And the hard part of this is that there are a group of people who are so focused on what they can write, rather than how do you support this in production?

00:33:43: I'll give an example which goes back to your saying.

00:33:46: What's the consequence of it and something that we need to get serious about, but I'll give you an example of this.

00:33:57: This happened two weeks ago and i travel almost every week.

00:34:02: I've got a trip coming up in September, and I think you and me are going to meet during one of those trips.

00:34:08: And so had made the reservation for this about literally two months ago.

00:34:13: So I'm usually proactive at making reservations.

00:34:16: So have made the September trip reservation back in June or July time frame—I can't remember.

00:34:23: This was two weeks ago Monday morning —usually a busy time—and get an email from The Airline saying we have, you know here's a receipt for your travel.

00:34:35: I'm like gosh.

00:34:35: why do i have a receipt?

00:34:36: For our travel and it's the same Monday morning.

00:34:39: so I ignored it right another few minutes ago.

00:34:42: little background thread in my head is like Why did they send that email to me?

00:34:46: And then look at it and I'm just glancing through it!

00:34:49: I don't know what they sent but its a receipt about ten.

00:34:54: Take a look at why they sent you that email.

00:34:57: This is where the luck plays a role, but even I got this emailed right and i'm looking at it And You'll be surprised!

00:35:04: I have A flight from The US to Europe... ...and then connecting From one of the europe destinations To Oslo.. ..And Then I Have A ticket From Oslo to.. ..You know Bell Brussels.. And Suddenly I have a Ticket From Washington Dulles To Denver Where I Live that segment from Brussels to Washington DC is missing.

00:35:29: I'm like, how are we going get from Brussels?

00:35:31: To Washington D C?

00:35:33: right well one thing good about me as a very disciplined I save every single conversation with the airline right.

00:35:40: so i go to my folder and look at email.

00:35:44: when they did their reservation on it.

00:35:46: compared what he got in that segment is missing.

00:35:49: so really call the airline and let's put something out here.

00:35:54: Well, gosh we that's really bad.

00:35:57: We can see your reservation is you know split and there's a segment missing.

00:36:03: I'm not kidding with you.

00:36:04: Wow

00:36:05: literally it literally right.

00:36:06: i'm not exaggerating.

00:36:08: It literally took on the Monday morning Two-and-a-half hours.

00:36:13: Yeah

00:36:14: two one half hours On the phone.

00:36:16: Once they fixed it reissued my ticket And put me back on my seat.

00:36:23: I asked them one question.

00:36:24: Thank you for fixing it, could you please tell me what happened?

00:36:29: Yes!

00:36:30: You know the response was we have no clue what happened.

00:36:37: So here's a question from you.

00:36:39: right We know this as software developers.

00:36:42: What are things that come to your mind?

00:36:44: Observability Faceability Accountability And this is a major airline.

00:36:53: I'm not going to embarrass them by saying the name here, um... This was one that I fly all of time and they have zero clue how these changes happened.

00:37:06: The cost of failure right?

00:37:08: Now it's just for one customer.

00:37:10: but imagine if there were repeat patterns or several changes made.

00:37:16: Imagine the costs that are incurring And the cost of, what if a seat is not available on that flight after all once they removed this particular passenger from it?

00:37:28: I think that's where we... It's far more than developing code.

00:37:34: I was talking recently to somebody and they said oh you know what i'm super excited.

00:37:38: my developers writing code with AI because feature after, future-after-feature.

00:37:46: And I said hang on a second.

00:37:49: i'm surprised you say this because application development is not about delivering features it's about supporting in production.

00:37:59: have you thought about scalability?

00:38:02: Have you taught about performance?

00:38:03: Have we thought about security?

00:38:05: ?Have you talked about reliability observability traceability?

00:38:09: if things go up what is going to be your response?

00:38:16: Are you going to sitting there happy that your developers are producing code at a fast rate, or are you're going to being happy that you put in discipline along the way.

00:38:25: To guarantee that what they creating it's actually sustainable as well?

00:38:30: and I think this where i think we have had free ride for long time of our industry.

00:38:40: With the speed at which AI is able to generate code, it probably will come to a point where we finally have to show that we are not just fast but we're responsible as well.

00:38:56: And here's the hard truth about this.

00:38:59: I think either we grow up or others will show us how to grow up.

00:39:06: You know, this kind of goes back to my time as a child when I behaved more responsibly.

00:39:15: My parents had to do less parenting When i was completely out of control.

00:39:21: they have to step in and parent me.

00:39:25: the lesson I've taken, right?

00:39:27: If we grow up as an industry... We need less parenting.

00:39:34: But if we don't show that discipline eventually I think the lawyers and the regulations, which is not going to be pleasant.

00:39:43: I would rather self-govern than being governed by people who are probably not as good at governing us.

00:39:51: but that's a bit of dark side where we're in right now because this isn't sustainable for an industry especially with and to a greater extent, we need to really start relearning some of these sustainable practices I think.

00:40:16: Could not agree more.

00:40:19: there's so much opportunity now Especially now when we can build code faster than ever before, that were building things that make sense.

00:40:37: We're building.

00:40:37: the right thing is that we are applying ethics to consideration of what we built.

00:40:43: It's so easy today for us To Build systems of oppression into our software For example if not careful and we don't want to do that at massive scale Right?

00:40:59: Like thats bad

00:41:03: And to be able to recognize that, and I think the challenge today is I once posted on social media, when the ignorant get excited.

00:41:21: The intelligent becomes frustrated.

00:41:25: so you look at this and say oh my gosh!

00:41:28: You are so excited about this but it's kind of like taking a chainsaw and giving it to an random person, right?

00:41:38: Sure.

00:41:39: A chainsaw is going make somebody productive but may not be that person who's wielding at this point.

00:41:46: And I think the key for us to realize power of these tools But use them in safe way Is essential

00:41:57: Absolutely So.

00:41:59: one of ways Yeah, I can potentially help us and this is something you've also talked about at length And it just also touches on what we've talked about earlier Which is there were things about agile that?

00:42:13: We really weren't able to scale out because it was the constraints Were not there to allow.

00:42:21: That one of those things from my perspective Is to really do TDD.

00:42:25: well right It wasn't extremely hard when doing it manually.

00:42:29: And you've talked a little bit about using AI to support TDD, so how are YOU doing that?

00:42:34: Because...using AI to help with testing and verification and make sure we're building the right thing feels like good use if we can do in a way that's reliable and sustainable.

00:42:44: I'm really interested your thoughts on how to do that.

00:42:48: Absolutely!

00:42:49: That is super-good because one of things i realized was creating software.

00:42:59: But fundamentally, at the end of day with all respect I'm not a programmer and problem solver that you know.

00:43:07: programming is part of the way i solve the problems but it's not an entire thing to do.

00:43:13: And one of my frustrations have had.

00:43:16: if people see me doing live coding they can type really, really fast.

00:43:22: Or as my wife likes to kind of put me where I belong she would say you can type really fast backspaces sure right?

00:43:30: I'm very fast in typing but But what i've realized over time is that As fast as I can type translate my thoughts into code much faster than I can ever do.

00:43:53: Sure, you know i started programming when there were no IDEs.

00:44:01: just look at me and say what do you mean by that?

00:44:04: So all it had literally was vi.

00:44:08: an entire development was done using vi right.

00:44:11: That's how I started forty years ago When he started writing code.

00:44:15: And the problem is Well, we then had you know IDEs.

00:44:21: We have refactoring capabilities it's ability to deal with multiple projects.

00:44:26: So so we saw this real improvement in our ability To deal With high volume of code and I'm Then Of course now AI is able to generate that code You Know quite fast as well.

00:44:38: so so too mean I know what quality is, but i can remove that burden of having to physically type every single character.

00:44:49: That he's going from the thoughts in my mind too.

00:44:52: do the file that are created.

00:44:55: But uh... What have realized?

00:44:57: in this process is that I still need to guide AI on what's the quality of a good test.

00:45:05: So, so i have created a set of skill files where I tell AI here are the do's and don'ts with respect writing test cases you know?

00:45:17: And then certain things tend to teach AI about it.

00:45:28: Your tests have to be cohesive, your test has to be really small, you're a test that's been focused on what the outcome is.

00:45:37: don't make your test brittle by dealing with the internals.

00:45:42: and for me good test drives better design.

00:45:47: unfortunately though because there an inference engine tends right code like most people written which probably not the best way to design code.

00:45:59: One of things I often tell developers is, reduce state in your code as much as you can?

00:46:05: Well that really comes from their test again.

00:46:08: so when they don't give any information like go to AI and say write a test The test is utter rubbish because what it's doing is creating an object injecting state into it and verifying.

00:46:20: And i just show them this type beginning, and I often encourage them not to.

00:46:27: I've written tests like this twenty years ago but have learned to do better!

00:46:32: And so now i direct AI to make the code much more stateless as they write that test.

00:46:41: So these are lessons.

00:46:42: im taking back in asking a question How would I write Test Cases?

00:46:46: How Would I Direct my team To Write Test Case?

00:46:50: Now I can Do The Same Thing coach it to write better test cases, and then come back in review with.

00:46:59: And when I see that there are flaws Here is an example of the code that you wrote, and here's why this is a terrible thing to do.

00:47:11: And here it was an alternate way to write it.

00:47:14: now when I feed that back?

00:47:16: Can see that AI is changing that in evolving.

00:47:19: so too C that AI Is able to right better test incrementally evolve with ease away for us to improve hand.

00:47:28: The beauty of this is This something we can share where the rest of our teams not just within our own team, but across different teams and organization as well.

00:47:38: So I think over time...I would argue these things are also things that should go through a proper full request right?

00:47:48: Right?

00:47:48: so you're going to develop this knowledge of skills And then Go through the process of evolving it sharing get on Then coming back end improving It.

00:48:01: one thing's unfirmly in belief is It's a two-way process, right?

00:48:06: You don't want somebody who writes this and everybody else follows it.

00:48:09: That is not effective!

00:48:10: you want people on the ground Who are continuously learning from this And they create change requests.

00:48:18: Somebody else then reviews that makes improvements Then shares them back.

00:48:22: I think that becomes A richer experience for The team as well With proper effort Just like how Coaching your team to write better tests.

00:48:34: I think we can coach our AI, right?

00:48:37: Better test incrementally and that is useful because it's removing so much of manual effort on the part.

00:48:46: yes And i get the results Of better quality at a faster rate With less effort over time which Is really what you want.

00:48:57: even one speed.

00:48:59: You want better quality at a greater speed.

00:49:02: Yes, exactly!

00:49:05: So what would you say is the next big change in development that your looking for?

00:49:12: What are you keeping our eyes on process changes or technological changes to look out?

00:49:22: I think what's going to happen is we're gonna go through this hype cycle.

00:49:27: And once they are on the other side, That these are not these two extremes.

00:49:35: where one extreme is where you know AI is not of any use don't bother with it.

00:49:42: AI Is not going to end up With some of the other things we have seen in the industry.

00:49:48: We're no over-the-past, you know few decades Where we said?

00:49:51: You know before gls or no code?

00:49:55: Yes, they all had marginal impact but not a significant impact.

00:50:00: I don't think AI is going to end up over there.

00:50:03: But on the other hand i also Don't Think AI Is Going To End Up In This Other Aspect Where You Know Hey We Don'T Need Programmers Anymore The Business Can Simply Sit There And Tell AI What To Do and It's Gonna Get Things Done?

00:50:22: that's not going to be the case.

00:50:24: AI, more and more.

00:50:27: I think we're gonna realize that AI is a tool That what it really is.

00:50:31: It's a tool And We are Going To Spend The Next Few Years Really Rethinking About What Does Engineering Mean With The Presence Of AI?

00:50:48: I'm still learning about these things.

00:50:51: To me, there are a few things we focused on in the past that we may end up focusing less on.

00:50:59: with all honesty if you ask how important is design patterns going to be?

00:51:04: My argument this it was not as important even before though we did place more emphasis on it and always felt better communication tool than designed tools but I think it's going to play a lot less role moving forward.

00:51:20: How important is other aspects of development, verifiability?

00:51:26: Verifying the modularity of applications?

00:51:31: how do we write better set-up test cases and bring back more discussions too?

00:51:40: what is the correct outcome?

00:51:42: so This finally probably helps both, in a way I would say that automated testing did not eliminate testers.

00:51:54: Automated testing finally made testers focus on what they should do and take focus away from where we're wasting their time on it.

00:52:05: And i'm going to say AI is not taking away programmers.

00:52:09: it probably is going to make programmers actually focus on the programming they should think about and remove the drudgery of process in the middle, right?

00:52:20: I think we're gonna begin to realize more and more off that.

00:52:23: And as a result i think organizations will start truly understanding what using AI means to them core competencies into their teams so that they can leverage AI a lot better.

00:52:43: I am thinking it's the next five years from where we are, We're not out of this hype cycle yet but beginning to see that right now.

00:52:52: maybe another year or two and then uh i think we will have the shift back to the middle Where we begin to realize The power off This tool And Then the capabilities redefine what programming is.

00:53:11: We're not going to remove programmers, but we are gonna transform programmers and I'm excited honestly for that!

00:53:18: And this where the relearning will happen in our industry... To me a bigger risk if from what i've observed so far AI is an amplifier So If you don't know You'll get results out of it.

00:53:37: What that means is if we are ignorant of this right now, five years from now.

00:53:43: We're going to have a break in pipeline where not gonna have suddenly the junior developers wake up and become seniors.

00:53:52: but here's an impact That bothers me And I'll tell you a bit of pessimism on my mind But i think it's more realistic as To what's going to happen because unfortunately This Is The Reality Of The World We Live In.

00:54:05: So so the current trend is we are focusing on senior developers using AI.

00:54:12: We don't need these junior developers, right?

00:54:14: We do not need to train

00:54:15: them.".

00:54:16: What's going to happen in this?

00:54:17: people who are fresh out of college or go through school.

00:54:21: you're gonna have two groups if you will.

00:54:24: unfortunately a smaller group off.

00:54:27: students and young developers were gonna realize This Is Not Gonna Be The Way To Go To the Future.

00:54:35: they're going to apply their critical thinking.

00:54:39: They are gonna spend there time mingling with experienced developers, reading and learning an experimenting not just throw things into AI and get the results.

00:54:52: The second group unfortunately a much bigger group is going be people who don't get it.

00:54:59: Who's just throwing on AI garbage in garbage out?

00:55:03: And shift forward four or five years.

00:55:06: We will, as an industry suddenly realize we need people who are very knowledgeable know the fundamental.

00:55:15: Here is the unfortunate twist of events.

00:55:18: when that situation happens you got a smaller pool of good developers or young and not super experienced but capable on a vast number.

00:55:35: bigger corporations, which have bigger pockets will grab that small pool.

00:55:43: Which is capable?

00:55:45: so those people would have a good time and they'll get hired.

00:55:49: but the rest of the organizations That can't compete for their pool or The ones are gonna end up suffering.

00:55:57: yes.

00:55:58: So this is the irony of the world right.

00:56:01: we all contribute to this problem But we all not face the same consequence, right?

00:56:07: Because some organizations will still come out winning because they have a bigger pocket and They has the ability to move past their mistakes even though they contribute or do it.

00:56:21: But the rest of the organizations are not going to be that fortunate.

00:56:25: I think this is gonna be an overall, you know shift both in the way we embrace technology but also the way re-learned to rebuild our workforce.

00:56:37: Uh...I think it's a lot change uh..but in all honesty i would say there two ways.

00:56:44: look at almost every journey which involves uncertainty is painful if it's hard for us to swallow.

00:56:56: It's not as painful, If you're going to be able to sustain and come out on the other side right?

00:57:04: So The people who are financially You know okay may not be enjoying that time but they will get through.

00:57:16: And the people who are not financially stable for various reasons, probably will go through much more hardship during this time as we come out of it.

00:57:25: But but the good news is when you come on to other side I think what excites me Is We Will Re-Learn and I Think Better Days Are Ahead Even Though It's Kind Of You Know not as clear when we look at things given the market right now, perhaps super optimistic for what's coming out on the other side.

00:57:51: I certainly hope you're correct!

00:58:11: positioning yourself well in a situation like this big AI boom is to be aware that you are in a hype cycle and behave accordingly.

00:58:22: The second part, it's even more important not being denial.

00:58:31: I often jokingly say There are fifty one states in the

00:58:37: U.S.,

00:58:38: and most people live In The Fifty-First State, And People Look At Me Puzzled And Say What Are You Talking About?

00:58:44: I say The Fifety First State Is Called The State Of Denial Which is a scary part right.

00:58:52: so A Lot of Us Live In The Stateof Denial.

00:58:56: That's Where The Most Population Is.

00:58:59: We Need To Get Out Of That StateOf Denial.

00:59:04: Venkat, thank you so much for spending today with me.

00:59:06: This has been a fantastic conversation and lots of great insights from here.

00:59:09: I greatly appreciate it.

00:59:12: Michael Thank You For Having Me.

00:59:13: Is

00:59:14: there is there way would like people to be able To follow you or?

00:59:18: Or an event coming up where People can meet Up With You?

00:59:21: Oh, I'm in Europe almost every other week if not more frequently.

00:59:26: So somebody is following me on social media.

00:59:29: they know i am constantly a user group or our conference so I appreciate that.

00:59:35: but actually post where I am

00:59:37: fantastic.

00:59:38: well very good thank you very much

00:59:41: cheers.

00:59:41: Thank You so much.