Insight ON You Can No Longer 'YOLO Mode' It. Here's How To Adapt to GitHub Usage-Based Billing.

Audio transcript:

You Can No Longer 'YOLO Mode' It. Here's How To Adapt to GitHub Usage-Based Billing.

Sumner Ohye (00:02):

I have responded to emails this last week where people hit their budgets and what happens? None of their devs can access Copilot. We talk about how impactful it is to their day-to-day operations now. They essentially couldn't work. It's the same as them not being able to ship code. And then the result is, well, let's just take off all budgets as a solution and just go for it until we figure this out, which is not sustainable for them.

Jillian Viner (00:26):

Welcome to Insight On, the podcast where you get real insights on technology decisions from people who are actually making them. On June 1st, GitHub changed how Copilot gets billed. Flat rate seat licenses are out and now it's metered by token, by model, by task. And if you've been paying attention to the developer forums, you've seen the reaction. Preview invoices showing 10, 20 times what people expected. Developers asking whether one agentic session just wiped out their whole month. Engineering managers scrambling to figure out how to set some guardrails before that first real bill hits. But here's the thing, this isn't just a pricing story. It's a forcing function. The billing change is making organizations confront questions that they should have been asking all along. How do you govern AI agents operating inside your code base? How do you measure whether identic workflows are truly delivering value?

And how do you decide what's worth delegating to an agent versus keeping with a human developer? I'm Jillian Viner, and today I'm bringing the contentious questions from the forums to get answers from two people that are living inside this transition. First, a GitHub sales leader who's been fielding those hard questions from customers since day one and an Agentic field CTO from Insight who's helping teams through his own solutioning. All right, let's go. Sunner, before we get into this, tell me a little bit about your role here at GitHub.

Sumner (01:59):

Yes. So I am on our services team here and I work with our partners providing services for our customers in Amera and APEC.

Jillian (02:08):

So you're fielding questions and calls from partners like Insight and also directly from customers?

Sumner (02:14):

Both, yes.

Jillian (02:14):

Okay. How spicy have those calls been lately?

Sumner (02:18):

I think it ranges based on who's in the room. Obviously in a smaller setting, you get some pretty candid feedback. And there's also some email threads where I think sometimes when people don't know you, the feedback can be pretty radical.

Jillian (02:31):

Oh, so you have dozen thick skin in your role then.

Sumner (02:33):

Definitely.

Jillian (02:34):

Well, there's a reason I think for some people to feel a little bit spicy right now. The developer forms have been quite noisy since June one when some pricing changes took effect. In fact, I was kind of cruising Reddit. I love to do this to find out what is the technical chatter. I came across the post where one user obviously seeking help, part help, part perhaps venting, that their current usage for a process that they had with GitHub would increase 24 times. The cost of this would increase 24 times just continuing their current workflows based on the new billing structure. That seems massive. That would send some shockwaves. So explain to me what did happen on that June 1st announcement and is this particular user experience the norm? Is there an adjustment needs to be made here? What's going on?

Sumner (03:25):

That is a great question. And if I had a 24X change or forecasted change in my bill, I'd go on Reddit as well. So let's split that question up. Is that normal? No. Most of our users are not going to see a 24X change in their bill. Now, will users see a drastic change from what their billing was pre-June 1st and post June 1st? Absolutely. If their behavior doesn't change, their bill will increase at some sort of, not necessarily multiplier, but as we were calling it, usage, depending on what models they're using, how they're using it and how much you're using it.

Jillian (04:02):

Explain just very simply what was the change that took a place for June one in case someone is not paying atention.

Sumner (04:06):

Yeah. Let's think of it this way. Have you ever been invited to a wedding that's an open bar?

Jillian (04:12):

Sure have. My favorite kind.

Sumner (04:13):

Favorite kind, right? You're planning an open bar. Usually there's a cap on it. GitHub Copilot previous to June 1st, we had PRU, so premium requests, but if we take a step back before that, we had an open bar for Copilot. So whatever you wanted to do, you could do, which was really fun.

Now from a business standpoint, as the models have changed, as the costs have changed, that was not a sustainable model for GitHub, and that's not where the industry went. So where we went post June one, June one, you get a ticket. And after you use your tickets for your drinks, you pay for what you want to consume post that. So it makes sense. It's not anything new, but I think experiencing the change the first time, it's like, wait a minute, I was used to an open bar and now I have to open up my wallet to buy additional drinks.

Jillian (05:03):

Yeah, I was having a great time before and now the party's over. I got to be responsible here. That sounds very reasonable. And I love the analogy that you shared. I think that's very practical. Announcing this consumption meter 30 days before it kicks in maybe caught some people unprepared. And you even said it yourself, there's probably behavior change in there. We're forcing a different function. So now that teams have had kind of a minute to understand like, oh, open bars closed. I got to open my wallet. What do organizations or what do developers need to do right now?

Sumner (05:38):

We made the announcement and we did have feedback that it was sudden. And that comes when you are a platform, when you are a home of developers, people live and breathe on GitHub. So they did put in a transition period that starts June 1st and runs three months. And then post that three months will be fully into our new usage-based billing models.

Jillian (05:57):

Okay. So they had some time, but it's still a bit of a shock. So people had to absorb that immediately. What is it that they're absorbing? What's the reality on the ground?

Sumner (06:05):

I think the reality on the ground is if we think about true enterprises, they budget annually and now you have a change that is a line item that they didn't have planned for. And to the previous example, for some of our customers, it'll be no impact. For other customers, based on how they're using it, it can be a sizable impact depending on their budgets. What they need to understand is how the billing works and who their heavy users are.

Jillian (06:32):

So in other words, I'm at the wedding.

Sumner (06:34):

You're

Jillian (06:34):

At the wedding. I've got my drink tickets. My date doesn't drink. Maybe they're going to have one drink. I can take their drink ticket.

Sumner (06:41):

Exactly. Yes.

Jillian (06:43):

Okay.

Sumner (06:44):

If we're to take that a step further, I want to know what the rest of your evening's going to look like. Is there a dance floor? So it goes back to the conversations we're having two years ago about ROI. So who are using these tokens? What are they using them for? Are they using the right models? For before, it didn't really matter what model you use. I want the biggest fastest with the biggest context window. Now it might be a second to pause and put more reflection on what model do you select for which task.

Jillian (07:14):

Got it. That kind of speaks to another piece of skepticism I've seen circulating around the message boards. It feels like it took a hot minute for people to get comfortable with this concept of AI and doing the coding for us. Now we've moved on. I don't know any developer who's not using AI to generate code, but it is kind of the question is identic coding, are those sessions genuinely better than if you would just do it manually yourself, especially if it's going to cost a lot? Where do you draw the line when you're thinking about checking code or building the code? If it's something a developer can do, how do you decide that? When do I make the AI do it and when do I just do it myself?

Sumner (07:59):

I mean, I'm going to get thumbnailed if I say AI is genuinely better than an engineer. I think that's a very loaded question. I think we're always looking with an engineer. When you make that decision, I think if you are a company and every company is going to have technology, every company is going to have engineers, whether it's internal or external. And if I were to take a pause, look around our house versus our toothbrush, our alarm clock, lawnmowers, bread machines. Everything is going to have technology and be connected. And for that, it needs to be supported by developers. So then we say, okay, what developers should be using AI? I would say all of them. You're able to go faster. You're able to be more efficient and more secure. Now, if you want to just hit a button and not use AI, but just push it forward, I don't think that's the right use case.

And I don't think you'll ever get the results you're looking from from a leadership if you're in the CTO, CTO or CIO seat.

Jillian (09:01):

Got it. So you said AI should be there, but it should be a matter of looking thougtfully about what you said earlier. What's the right model to use? How big is that context window? These are the decisions that people now have to reevaluate.

Sumner (09:13):

Absolutely. And I think what are you doing? Are you supporting internal products, external products? Are there timelines around that? Is there revenue for the company? Is there a risk and compliance? Is it a push or pull kind of decision?

Jillian (09:27):

So I'm going to ask you something a little about the governance of this because we've established Agentic is part of the work developer workflow now. Agents are out there. I saw something really clever. We're in San Francisco obviously at the GitHub headquarters right now, and I saw a clever ad the other day. I mean, AI ads are everywhere, but this one, I won't reveal the brand. It said no more secret agents. I was like, dang, that's good. GitHub released your agent panel back in October last year that gave some visibility for different reasons, but now it feels like something like that, knowing what agents are in your org and what they're doing seems like even more important than ever. I

Sumner (10:06):

Think that dashboard had perfect timing, probably could have come earlier. Maybe perfect timing isn't the right place for it. It is very valuable to know what's going on within your org. I think the security question that you're leaning into, it's still not enough. What I'm concerned about is when Copilot or AI tools get in the hands of people that aren't developers, to our knowledge workers.

Jillian (10:29):

You mean myself? I dangerous.

Sumner (10:32):

And it asks you for a token or it asks you to connect to an MCP and all of a sudden we have no visibility on what's going on behind your harness. And those are things I think even GitHub and other industry leaders need to raise because those are the things that should be keeping security teams up at night.

Jillian (10:51):

Does that also impact pricing and usage then? I

Sumner (10:55):

Would say right now I don't necessarily see that or see an impact from that coming, but I do think that there could be additional products that look at agents running across an organization, whether that be GitHub or outside of their dev tools.

Jillian (11:13):

Okay, that makes sense. So the June 1st change happened, you're fielding questions already. What's the number one question that you're getting from users?

Sumner (11:24):

I would say we're early in it. So we haven't seen a full billing month. I think if we had this conversation in July, what those conversations in July look like would look a lot different than in June. I would say right now the biggest questions I'm getting is what is my bill going to be? And did we set up our billing and threshold limits correctly?

Jillian (11:49):

How do you do that? What advice do you give them?

Sumner (11:51):

There's documentation, which everyone loves, right? I'm going to go read documentation.

Jillian (11:55):

I'm going to have Copilot read the documentation for me and ask the questions.

Sumner (12:00):

I think at scale it's working with partners. I think when we're reactive, we try to get on calls. I have responded to emails this last week where people hit their budgets and what happens? None of their devs can access Copilot. We talk about how impactful it is to their day-to-day operations now. They essentially couldn't work. It's the same as them not being able to ship code. And then the result is, well, let's just take off all budgets as a solution and just go forward until we figure this out, which is not sustainable for them. So I'd say work with your GitHub seller. We have solution engineers, but really every customer should have a partner that they're going to for though leadership and guidance that matches our roadmap and our releases.

Jillian (12:44):

So it's not going direct to GitHub, it's finding a third party to help be your consultant mediator, take a look at what's on the table here in terms of your agreements and help create, I guess, workflows for a more predictable billing?

Sumner (12:59):

Absolutely. You said it better than me. So I think with that too, a lot of times at GitHub, we have a platform lens. We're looking at actions, we're looking at security. We're looking at how do you test? How do you get into production? But when you bring in someone that's third party, they can look beyond GitHub. And as we've said, these tools and these agents, they're going beyond just what you're doing in your DevSecOps practice. And I think that's why it's valuable to have a partner that can look at really your entire ecosystem.

Jillian (13:28):

Yeah. What do you anticipate in that conversation looking like in July? Do you think that's going to be even more of a wake-up call adjustment for people?

Sumner (13:38):

I think it'll be interesting. I think if we look back when Copilot launched, every conversation we had was, is your model getting trained off of our data? Is it secure? What's happening? And we had people leaning into that. Then after that it was, okay, it's secure. We get it. We own the code that we ship, but now what's the ROI? And we looked at Dora, we looked at space, we looked at certain things that were maybe a little fuzzy at best. Measuring productivity, measuring lines of code, measuring releases, looking at things of speed. I think now that we know GitHub Copilot is secure, we know it works, we have new models. We're going to almost take a step back and say, let's look at that ROI conversation again. The other thing I think we'll see a lot of conversations around are sharing best practices within organizations where with GitHub Copilot, it's so intuitive that end users, both devs and knowledge workers are solving for their little paper cuts that are so small that it wouldn't make sense for an internal team to support it.

And it's so small that they wouldn't hire someone or subscribe to some sort of SaaS product. How do we get them best practices so that they have guardrails, they have skills like agent skills to build what they need in a repeatable way that makes sense for the business?

Jillian (15:07):

Yeah, it's a wild time. I mean, truly from a knowledge work perspective, it's so fun and crazy to be able to look at my own workflows and be like, "Oh, I can just fix this. I can just go build something to do this for me. And I can talk to IT people, I can talk to developers for their advice, but at the end of the day, I can just go build something. And if it breaks, it breaks. And I can keep evolving it. I think the speed of the evolution is the thing that's hard to keep up with. And credit to Microsoft, there's been several instances where I want to go do something or have an agent do something for me. And maybe I try to go build it and I swear the next week it's there. Copilot's already done it, so Bravo to Copilot and all that.

But I love the empowerment that's happening. It's going to be an interesting few weeks as we get through this billing cycle. What's the next milestone or potential hiccup in the road that organizations need to be bracing for just kind of thinking ahead?

Sumner (16:09):

I think it goes back to that middleware. I think if we look at OWAS top 10 for MCP two years running now, we're looking at injection threats. I think they are things where people are going to trust their agents so much that they will put tokens in the chat and there will be bad best practices. And for the most part, you might say, okay, it's localized. There's firewalls. No one's going to access that. But how do we have better security around AI in the middleware and more visibility to security teams?

Jillian (16:48):

Good advice. Good things to be looking out for. Any final thoughts on how to help organizations right now adjust to the billing? Or actually, let me ask you this. GitHub is fundamentally changing how software development is happening. What are you most excited about for software development going forward, especially as all this agentic stuff continues to take off? Oh

Sumner (17:11):

Man, what am I most excited about? I will tell you why I interviewed at GitHub. Okay. GitHub gives the power for anyone in the world. And this isn't their slogan, this is my take on it. You can get enterprise tools on a credit card monthly. Full stop. Same stuff the Fortune 100 companies get. I think with what's happening in AI, we're lowering the barrier for people to test out ideas. And I think over the next six to 12 months, we'll see so many things just get tested that were just on a notebook or on a whiteboard somewhere that would've never got the investment or time of day. And these aren't going to be big companies. They're going to be people doing it as passion projects. They're going to be back in the garage building. And I'm really excited to just see what comes out and what sticks.

Jillian (18:07):

Love that. Okay, so now with this billing change, I have to ask you kind of a spicy question. Why would organizations then choose not to run more of the AI modeling locally?

Sumner (18:20):

That is a great question. And I think a lot of it will depend on what really the size of the organization is. So it will mirror the same conversations we had when it was should we stay on - prem or should we go to the cloud? And it depends on that CapEx. We already know what the shortage of GPUs are. We know what the refresh cycle looks like. So if you look at just a running cost, sure, it could be cheaper to host yourself. If we look at total costs, for most users, it's not going to be a cost savings. If anything, it's going to be a negative. Now the segment also is important for the person that was on Reddit that went 24X. They're probably doing a lot of things.That person probably has the ability to host a box themselves and have a cost savings.

They can maintain it, one person team, small team. They kind of know what their security risk tolerance is, and that's that. Once we go into enterprise, who's going to maintain that? Who's going to update the models? What sort of security risk are you going to introduce by maintaining yourself versus going with a provider? And we have to look at that whole conversation versus just looking at a run rate on a monthly bill.

Jillian (19:36):

Well, Parker, thanks for being here.

Parker Johnston (19:37):

Always.

Jillian (19:38):

You've been on the podcast before, but just for people who don't remember who you are, you're kind of famous, but no, it's really good to have you here. Tell me again just a little bit about what you're doing at Insight.

Parker (19:48):

Yeah, sure. I'm an Gentech field CTO at Insight. So really spending that time out in the field with our customers, talking about their challenges. How can we help them use agents to go and be successful? How about adopting some of the AI strategies that maybe are going to drive those business outcomes, opportunities and capabilities across their entire enterprise? Traveling all the time, trying to go and see those real-time use cases as we go. But it's really just also the love of developer productivity and experience that's kind of led us to the conversation today. I think of leading some of the partnership with GitHub, how do we have these conversations and really help to make sure across the platform we're making the most out of what we're already paying for.

Jillian (20:25):

Yeah. Speaking of that, how much have the conversations changed since June one?

Parker (20:30):

There's been a large change. It was all you could eat, everyone's showing up, 300 premium request units. I'm going to make one really good prompt, send it, we'll just wait 30, 40 minutes for it to be done. But I think now as this June 1st change for usage-based billing has come about, it's now been more of a conversation of how do we control that spend? How do we start to make sure that we're shifting from budgeting to forecasting properly and making sure we're setting the proper security guardrails in place so that there's not an unexpected bill coming about? We're getting phone calls. I'm nervous for what's to come. I'm really not looking forward to the end of the month bill that we're going to get. So we've been meeting partners and customers throughout that journey to make sure everything's properly set up and capped as needed.

And I think the other big benefit we're getting or the big question we're getting and the benefit Insight has to offer is how do I know what my team's using? We have done a lot of dashboarding and building some IP around saying, this is where your team is today. How do you want to allocate this to your heavy users versus those users that are already inside their AI credit limit that already exists? Before

Jillian (21:28):

We get into how do we address those things, let's talk about just adoption of GitHub. How has that really changed organizations on the inside? Where do they sort of hit the major roadblocks as they're trying to get more developers using it?

Parker (21:43):

Yeah, I think there's been a large shift in that GitHub enterprise adoption motion. It went originally from, hey, I have source code that I want to maybe move from TFS or some legacy source control system into more of a community aspect. I can now have disparate development systems and tasks going on, multiple people working on the same file at once. That was how it started. It's now the entire Agentic platform that I can hand things to an agent. I can have a business analyst getting well-defined work created with Copilot, but it's really now running my entire software delivery lifecycle. Instead of having to buy multiple different tools, I can just upload my code, ask questions, prompt as needed, handle my pipelines and deploy into something like Azure in a very simplistic manner. So that's really shifted the story there as we've gone through a lot of that motion together.

Jillian (22:32):

Are you still experiencing a little bit of resistance or fear about AI taking over developer jobs? I have to ask

Parker (22:39):

Because - It's an interesting question. You see a lot of models changing, a lot of new models always coming out. Methos is a big one everyone's talking about from a security perspective. I don't necessarily, and I always fall back to, I don't necessarily know if it's always going to take jobs, but I think it's going to make people think about their job differently. How do I go through doing orchestration is a very different conversation that everyone is having in the software delivery life cycle today as compared to when this all started. If you think back through GitHub Copilot coming out, oh cool, I can ask my code a question and get an answer. But now it's more of a conversation of, I want you to go and implement these features. That's Daisy chaining a lot of different capabilities together. How do we get the proper standards in place?

How do we build custom agents to handle it? What about custom skills that we maybe need to bring together? That's really where it shifted. So taking jobs, no I think it's going to be maybe pushing people to do a little bit better. And one of the most interesting things from Kyle Daigle, who is kind of a large

Influence at GitHub, he was the COO for a while, since moved into a new role. He wrote a really interesting article that made me think about it differently was it's not AI that's going to take your job, it's the intern because the intern has grown up in the age of AI. They know how to take all those different capabilities, smush them together and get the results that you need instead of having that AI fear of what do I do next? It's now, oh yep, fearlessly going in and doing it. It's no different than the early 2000s when computers became a big thing. Everyone shifted from desktop to laptop. Some people shifted from web development to mobile development. Same thing.

Jillian (24:13):

Yeah. Is it better though to truly have these AI workflows when you're talking about code? Is there ever a moment where as a developer it's just easier or more, I guess, sensical to just be hands on the keyboard and just do it versus going through the machine? I

Parker (24:32):

Think there's a couple benefits. You have to think about standards is a big one. Every organization wants to be security-minded, security upfront. How do we drive one organizational standard where if you have to come in and maintain my code,

It's not a mess. It is the standard we're all accustomed to doing so we can speed up that resolution if there's a potential outage and you're on vacation. That's the start of it. We can now drive standards, but we can also remove the boring work that I didn't get in the industry to do. I got in the industry to solve difficult problems, do fun challenges and be more forward-thinking where if I can move 75% of that work to an agent that's going to scaffold out my models, scaffold out my API calls, do my documentation and most importantly write my tests for me, I don't want to write those. So I can now go and do the frontier work out inside of a hyperscaler to be leading the industry or leading my business deliverables. That's where I see the value is. How do we offload the standardization work, the low level work and allow me to get back to what I enjoy doing as kind of that benefit.

I should continue to drive adoption end to end of the entire Agentic process.

Jillian (25:34):

It's a great vision. I agree with you, but it does involve trust and it's trust in a couple different layers. The developer has to trust that the AI is actually producing code that it believes in can follow. Whoever's overseeing that budget has to trust that things are going to maintain affordable, which is obviously a big question mark right now. And then there's a lot of security concerns as well. So how do you move forward in this new agentic software development lifecycle addressing those three layers of trust?

Parker (26:07):

We'll start with the pricing and the cost overruns. One is set your environment up properly. Take the time and the due diligence to make sure you understand what you're enabling, why you're enabling it and what it's going to be used for. I think if you don't understand that, probably don't turn that feature on straight away. Let's better understand what it's going to look like for our development teams, leadership, whatever it is, to go through and have an auditable process. Once we understand that, let's put guardrails in place for the security aspects. Let's turn on the security features that we have to make sure we're not pushing clear text passwords, connection strings, all of the information we don't want out in the ether. Let's make sure we're hardening that environment as well. And then as you're kind of moving deeper through this motion of, hey, let's let the agent go and do what we need, just don't forget that it's not an overnight turnkey solution.

There's a lot of work that needs to go into building agents, proper skills, proper agents, whatever it is properly with custom instructions, with the capabilities to not commit certain things and to act the way that the user wants them to do. But also as that's continuing to mature, you're going to have to audit it. We're going to have to go and build that trust a little bit more. Oh, maybe I'll let it take 82% of the work this time. 83% will slowly grow. Don't expect it to be perfect the first time. So slowly build trust and confidence at your enterprise level, organization level, that be you kind of inside the enterprise and at the development level.

Jillian (27:27):

Good. It feels like the AI workflows are now part of software development. It's embedded in Teams and for a long time it's really been a free for all. I think you used the buffet example. I've heard that one a lot. Earlier we're talking about open bar at a wedding. It's been a party. It has been. No matter what metaphor you choose to use, it's been a party. It's just been go - to's the biggest, boldest, most powerful model to do whatever you want and see what you get. Obviously the pricing model that changes and it's having to force new behaviors or practices. What are some of those behaviors or practices that developers need to start changing in their workflows? What are you observing and talking about with your peers?

Parker (28:07):

Yeah, I think the answer is you can no longer YOLO mode it. You kind of got to go in there and make sure that it's actually calculating what you need. The biggest things we're seeing are a couple, incorrect model choice. Maybe I'm looking for something like Opus48 I think is the big one that we're seeing a lot of files try to be edited by at this point to do markdown. That's not the solution you need for markdown. So let's go through and make sure we're scaling the proper model to what we need. Some of the metrics that I've seen recently are depending on the model used about one and a half million additional tokens get burned because it's the incorrect model trying to do an output. So you're now wasting five, $10 of additional compute on something that doesn't matter. Additionally to that, there is a massive cost savings, about a hundred X difference between a nano model and a full size model where if we're choosing the right one to do documentation, you're going to have a massive cost savings there as well.

So right model, right time, right place is a big one. But additionally, context, that's the big one. As you're starting to have these long running conversations back and forth with an AI system, your context window's growing. I know I'm guilty of it. I refuse to usually click the new chat button. I'm just going to have this context for the last three weeks I've been working, continue to grow. And before I know it, it's a million tokens. So every request I'm sending is now a million tokens. Development teams need to be trusting that if they click new chat improperly prompts, they are going to get the same result sets back. Again, it all comes down to the prompt style they're putting in there and the value in the prompt. Start the new chat, drive the value from there, and don't be afraid to ask for small changes, but make sure you're using the right model.

Jillian (29:39):

How do you make that change to a new chat and make sure that context is there? Because you're right, that's a... Man, it's all there. And why start from scratch?

Parker (29:48):

It's all of my information. I think that's the big one. There's features out there where we're all accustomed to inside of M365 Copilot, there's the personalization section. GitHub has a similar concept, but it's per repo. So instead of per user, it's now Per repo where I can go and set my organizational standards or my repo standards, and that context is going to help to derive the value and get attached to my request in a much smaller window. So now I can say, "Hey, every time I ask for X, Y, and Z, attach these rules to that prompt and I'm going to get a much more fine grain output. That's going to be much more helpful in my long-term development process." But also you can have a better prompt. Input tokens, sure, they're maybe a little bit cheaper than output tokens, but just make sure you're giving it enough context.

I know I'm guilty of it again, change button from red to blue. Well, what button? What RGB value? What information do I need? And I think most importantly, one thing I didn't hit on is start to explore on - device compute. That's one area where people are starting to see some value of these. Grace Blackwell chips are coming to our desktops now. Microsoft has had a large announcement at build around bringing all of those things to your desktop to be successful, even into a laptop in partnership with Nvidia. Start to explore what that looks like for your organization. How can I maybe take some of the less complicated workloads, process them on my laptop currently with something like Foundry Local, something along those lines, and then last mile delivery, this massive payload out to GitHub Copilot to go and do the complex work from there. You're

Jillian (31:16):

Talking about on - device AI.

Parker (31:18):

Correct.

Jillian (31:18):

AI powered PCs, which we heard a lot about several months ago. And I think about the time people were like, "I don't need this. " But now it sounds like, "Oh, here's your use case."

Parker (31:27):

It's the future. So I think it's been a lot of industry standards are saying that's what's to come next. We have RTX chips from Nvidia get announced. They're now inside of or will be inside of most Microsoft laptops and desktops that are coming in the future here. It was a big announcement at Build saying, "We're going to give you 70, 120 billion parameters, 70 billion parameters on your desktop, on your laptop, find use for it, and development's a great use case for that.

Jillian (31:53):

" Okay. So you talked about maybe changing the way that your workflows, new context windows so that you're not having that large piece. If someone is just now sort of getting in the GitHub environment, what would be your advice of best behaviors to start with?

Parker (32:09):

Best behavior there, I think we kind of hit on it earlier. Start small. Let's set the budget. Let's make sure we're not overrunning the 1900 credits for the next couple months. GitHub's giving you that additional little bit of buffer to make sure we can learn together, but go and find what works for you. Everyone is different. I don't think there's one silver bullet right now in the world of agents to go and cure all. But if we can set the proper budget, make sure we're doing an accurate output as needed for our requests, starting new chats, getting that really good muscle memory or whatever you want to term it in when it's your finger on a laptop, but getting that good memory in place to know new chat, additional context, build out a robust workflow where you're able to now define the work better before you start it and you'll be set for probably an advancement more than you would anticipate over some current early adopters in GitHub Copilot.

Yeah,

Jillian (32:57):

Because you're almost retraining yourself.

Parker (33:00):

You got to break

Jillian (33:01):

All those bad habits.

Parker (33:02):

I mean, think where the tool came. You think probably three years ago, maybe three and a half, four years ago at this point it was GPT 3.5, then it was 3.5 Turbo. You could just simply ask it a question you thought it was fantastic. Now you're asking it to go and implement an entire API endpoint with testing included with all of this additional information that agents fired on GitHub.com spinning up servers there as best as possible and you've had to do nothing. It's just crazy how far it's come through pipelines, through local device inference. We'll see what comes at future conferences and future events, but I think there's been a massive uptick in those four years and I'm sure the future is not going to slow down.

Jillian (33:39):

That's right. That's right. And I have to laugh a little bit because it wasn't that long ago we were talking about token maxing or measuring token usage to see how well teams are adopting. And now it's like we have to do a complete about face on that because that's the last thing you want for your teams to do because then they're basically twiddling their phones for the rest of the month because they can't do any work. They've already gone through their budget.

Parker (34:02):

There was a good article. I laugh at the photo. I hope it was a meme. I don't know if it actually was a meme or not, but it was like

Jillian (34:07):

Upgraded, I'm sure.

Parker (34:08):

People were running around San Francisco and they wouldn't close their laptops all the way because they couldn't let their agent die. They wouldn't want to lose connection or clamshell their laptops. They were running around to them half open to let the agents keep working because it was token maxing or you have some of the reports like, oh, if we're not spending a half million dollars a year in tokens, that's not a good target for us as an employer or employee relationship. Now it is totally different. What can we

Back to doing that we did do as developers beforehand that we can hand off some of that work to cheaper agents that might be more beneficial to us as an organization? But I also still think the future holds large token consumption. If you start to look at the way the models and the costs are changing, there's still a benefit from what I can do with agents today and my teams can do with agents today as compared to the salaries of multiple people that I would have to rehire to do the same work. And it's again, one communal code base, one communal opportunity. So token maxing, great use case. Let's make sure we're still using the right model and supporting our teams as best as possible.

Jillian (35:09):

Yeah. You're kind of hitting on an interesting ROI angle as well. I mean, you talked about you and your team this small can do the work of a team that's this big. It's a sensitive conversation, but it is the reality. But to your point earlier, it's allowing you to do the harder, more interesting work and not the foundational stuff. It's interesting too that you think that the intern is going to, you agree with that article, the intern's going to come in. There's still a transition period to make sure that people are learning this the right way. But going back to the ROI conversation, how do we measure this right now? Especially as organizations are looking at their budgets and trying to figure out if we have to break habits and get developers and engineers to operate differently to get this under control, this isn't going to happen overnight.

Parker (35:58):

I think the ROI conversation comes down to the metrics that we all kind of are accustomed to growing up. The DORA metrics, meantime to commit, how fast and how much throughput are we having? Those are still metrics that may or may not apply if you think about it because you're going to say, "Hey, this code was delivered faster or it was 100,000 more lines of code because now it includes better and more robust comments." That's all adding into this. How do we do that metric? That's a little bit skewed. The biggest ones that I would look at are two things that I've really enjoyed conversations with customers around. The first one's actually employee retention. This is one that people don't think about often. It's am I giving my employees the opportunity to grow, be happy at work, maybe be done with work a little bit sooner where I don't have to have turnover and go find someone else because I'm giving them these AI tools?

Can we make sure that we're increasing our value as a business as we continue to grow? We can keep that team together, very cohesive team together and grow. The second one is churn. I think this is the quality side of agents. Again, a lot of conversation around quality. How do we make sure that we're not churning pull requests as much or we're having to roll back software as much? That's the other one I would look at. I feel like lines of code, test coverage, all of those things are kind of,

They're out there, but I don't know how good of a measure they actually are when it comes to productivity or business value. But employee retention's been the big one that I've been focused on.

Jillian (37:23):

I like that. That's so good. That's a new angle on there. I think that's an important one to cover. Give me something a little bit concrete. Agentic work is now part of the workflows. Agents are out there doing stuff all the time and it's now probably going to cost a little bit more, even if you're prompting correctly and all that. So do you have an example of something that you've done or you've seen a customer do where the ROI is still there? It still makes sense to have that happen agentically versus having a person do it.

Parker (37:54):

I think there's a really good use case and it was one that I've really started to talk more about is how do we define the work better? Business analysts, the industry, the tech industry as a whole, tech world is moving way faster than anyone can keep up with it, let alone our friends that maybe aren't as technically well versed of day-to-day process. I could sit in a meeting and say, oh, spin up a TypeScript API in Azure, put it in a Cosmos database, do all these things. Yeah, the technical person's not going to know. The non-technical people aren't going to know that. So we're going to take how do we empower non-technical folks to better understand and define the work? How can we build contact servers? How can we build additional MCP information that's going to allow us to give them the power to type that sentence of what they heard into a window and get a robust work breakdown?

I know we've talked about how do we feed that through with human in the loop reviews? We talked about security and trust earlier. Human in the loop is still a massive win. So how do I open a GitHub issue and go through an analysis, planning, security, architecture review phase work that's very detailed that someone has to go sign off on? It's still a mission critical thing for us to do day in and day out. But as we're able to give more contextual awareness and educational uplift to everyone across the entire stack, I think it's going to be helpful. So that's the big area right now that I've been focused on driving value around is let us understand what you've done previously so that we can make a more robust implementation plan and then hand that off to an agent at the very end. That way we know it's going to work.

We know what they're looking for. We know we have the right coverage and we're not wasting tokens back and forth trying to quickly do something. So that's dig in, build the end-to-end flow. And what I've seen from that is less churn. So we can get it right maybe one and a half times now instead of running back and forth five times. Going through those peer review steps is absolutely critical as well. And the one step that I've really enjoyed building is the CISO architecture review board agents that go within that. We let them own those. They know that industry best. They know the requirements best of what we need to build. So let's let them manage those agents with pull requests, their own updates, and we can feed them into our agentic workflow where we're able to meet that flow of, okay, hit security. I don't have to wait for a security meeting every other week to get ripped apart about how terrible my code strategy is.

It will fix it for me now and recommend change based on those standards. Then we can continue to drive value from there. So it might not be fully baked in or owned concretely by someone else, but it's us speeding up the work to get to where we need to be faster.

Jillian (40:25):

Yeah, it sounds like you're removing a lot of headaches along the way.

Parker (40:28):

A lot of headaches, a lot of embarrassment. I think the big other aspect of coding agents in general has been empowering that younger generation that has fear to put code out there or code into a public forum. Maybe it's a pull request, maybe it's asking for a code review. They can get it done locally on their laptop now and then publish it so that's not the concerns of I'm too timid to put this out there and get picked apart. It's all how we learn part of the industry. So let's go through that together and use agents as best as possible to build some trust and confidence in that one.

Jillian (40:57):

Code review automation is getting billed twice now. AI credits for the model and GitHub actions minutes for the infrastructure. It's a very specific pain point people are facing. So how do you account for that in this cost model?

Parker (41:10):

I think there's a couple bits of value that go with that. So we're going to generate the code and pay credits and tokens, and then we're going to go and try to validate that in this build out motion. Maybe we're going to use agents or something different. All the infrastructure that that current one runs on is on your laptop. We have a coding agent open, it's in the IDE, that's kind of going back and forth. But then as we move this out, it's now moved to someone else's infrastructure. So how do we go through and say, how are you going to not bill someone for using someone else's infrastructure? You're now using all the capabilities that are afforded to you by someone else owning the platform or owning SaaS product for the most part. That's my answer typically to it. But additionally, there's still a cost benefit.

You have the capability now to get a peer review, code review agent, code review, and make sure you're still sticking to standard. So I now don't have to have a whole bunch of QA or SDETs on a team. I can let my coding agent drive that standard. It's going to catch clear text passwords. It's going to catch dependency, mismanagement, maybe that I need to clean up. So what's the value proposition to a customer to say, "Hey, for fractions of a penny in the minutes this thing takes to run, we're going to make sure your environment still is pristine and secure." So I think that's been the big argument of going back and forth like, yeah, you're paying for it, but it's two different features in two different locations of which it's running.

Jillian (42:31):

Parker, thank you so much. I want to learn so much from you. You're one of our greatest practitioners here, so I know that everything that you say are things that you are actively doing, you're actively doing your peer group and having regular conversations with people that are really at the cutting edge of all this. So thanks for sharing all of your knowledge. Always.

Parker (42:46):

I appreciate it. Thank you so much.

VO (42:48):

Thanks for listening to this episode of Insight On. If today's conversation sparked an idea or raised a challenge you're facing, head to insight.com. You'll find the resources, case studies, and real world solutions to help you lead with clarity. If you found this episode to be helpful, be sure to follow Insight On, leave a review and share it with a colleague. It's how we grow the conversation and help more leaders make better tech decisions. Discover more at insight.com. The views and opinions expressed in this podcast are of those of the host and the guests and do not necessarily reflect on the official policy or position of Insight or its affiliates. This content is for informational purposes only, should not be considered as professional or legal advice.