A Masterclass in Engineering Leadership: Will Larson

This content provides valuable insights from Will Larson, CTO at Carta, on engineering leadership. Larson emphasizes the role of an executive in educating others about engineering, the importance of documenting business decisions and understanding context in new environments. He discusses the crisis of engineering strategy and shares his experience of leading engineering teams at various technology companies. Larson also talks about the importance of testing for conflict and assumptions when entering a new company, understanding historical decisions, assessing execution success, using benchmarks for performance evaluation, and applying past experiences to new situations.

How does it apply to you?

This podcast can be a valuable resource for anyone interested in software development or engineering leadership. The insights shared can help new leaders understand the importance of documenting business decisions, adapting to new contexts, testing for conflicts and assumptions, using benchmarks, and applying past experiences to new situations. It can also be useful for seasoned leaders looking to reassess their strategies or understand the reasoning behind past decisions.

Developer Checklist Refined

Engineering Leadership

Addressing Engineering Strategy Crisis: Develop a consistent approach to engineering strategy and establish clear metrics to measure the success of an engineering organization.

Understand and Adapt to Different Technological Contexts: Realize that different companies may have different technological strategies and contexts. Learn to adapt your approaches to align with the company's unique context. The expected outcome is a better understanding and respect for the company's technological strategy.

Identify Undocumented Beliefs in Technology Management: Be aware of undocumented beliefs about technology management that may influence decisions within the company. The expected outcome is a more informed decision-making process.

Business Decisions

Document Important Business Decisions: Ensure that key business decisions are properly documented. This could provide a reference for future decision-making, strategy reassessment, and understanding of past actions.

Assess Execution Success: Evaluate the effectiveness of execution against the problem and decision-making. If a task is taking significantly longer, investigate and rectify the issue.

Benchmark Performance: Use benchmarks to understand performance. This could provide insight into how well you are doing compared to others and whether your approach is working.

Create a Strategy Document: Create a strategy document using Richard Rumelt's structure: a diagnosis of the current state, guiding policies to interpret the diagnosis, and coherent action to implement the strategy. This will ensure your strategy is clear and actionable.

Learning and Growth

Apply Past Experiences to New Situations: Test whether strategies that worked in the past apply in a new context. Double-check assumptions and update your thinking or models as necessary.

Building Fundamentals in Early Career: Observe, receive feedback, and repeat tasks to build fundamentals in the early stages of an engineering career. This will ensure that you gain a comprehensive knowledge base.

Learning from Mistakes: Make a habit of learning from your mistakes and the mistakes of others. This will help you to avoid repeating them.

Team Management

Test for Conflict and Assumptions: Engage in multiple conversations to understand the unique reality of a new company. This could reveal biases and help align your strategies with the company's context.

Balance High Performers' Workload: Be aware of the risks involved in over-relying on high performers. Outcome: A more balanced and sustainable team productivity.

Expose Team to Reality: Shift from protecting team members from reality to exposing them to it. The expected outcome is to allow team members to process and adapt to reality over time.

Engage with Team Members: Engage with different team members and create a space for productive conflict. This will build trust within the team and lead to better project outcomes.

Communication

Understanding Communication Styles: Understand different communication styles when interacting with different groups. For instance, start with the conclusion first when communicating with executives.

Communicating at the Right Altitude: Ensure you provide the right level of context depending on the audience when communicating. The expected outcome is to improve the understanding and effectiveness of your communication.

Prepare for Questions: Be prepared for questions from executives who may consume slides quickly and have queries ready. This will allow for a smoother presentation flow and better engagement.

Manage Presentation Pace: Allow the listeners to dictate the pace of the presentation to prevent disengagement and distraction. This will lead to a more attentive audience and better understanding of the content.

Feedback Processing: Differentiate between nonsensical feedback and feedback where there's disagreement within a shared context. Ask questions and revisit later for the former, and discuss points live during the meeting for the latter. This will ensure that feedback is properly addressed and incorporated.

Engineering Practices

Respect Others' Context in Decision Making: When making decisions, respect and consider the context and experience of others, especially those with more experience in a particular area. The expected outcome is a more refined approach in decision making.

Create Written Engineering Principles: Write down engineering principles and strategies to improve clarity and facilitate discussions around strategy improvements. The expected outcome is a more transparent and effective strategy implementation.

Establish Financial Guiding Policies: Establish financial guiding principles and actionable steps to maintain them, such as holding monthly finance meetings to review expenditures. This will ensure financial stability and discipline in spending.

Decide on Resource Allocation and Decision Making: Determine how to allocate resources and make decisions in the context of your engineering strategy. This includes prioritizing projects and handling important decisions like hiring and technology adoption. This will ensure efficient use of resources and consistency in decision making.

Enforce Strategy, Frameworks, and Policies: Determine a model for enforcing your strategies, frameworks, and policies. This could be a team-based model or a top-down model, depending on your company's specific circumstances. This will ensure the effective implementation of your strategy.

Implement the Navigator Approach: Consider implementing the Navigator approach, where one person per business unit acts as the CTO for that area. This person is guided by a written strategy but can make exceptions if they see fit. This allows for more informed and context-specific decisions.

Understand the Complexity of Engineering Velocity: Recognize that increasing engineering velocity is not as simple as adding more people to a team. Understand that individual talent and abilities play a significant role in engineering.

Treat Engineering Projects Like Art: Accept the unpredictability of outcomes in engineering projects, similar to the creation of art. Outcome: More tolerance of uncertainty in project outcomes.

Avoid Overreliance on Volume-Based Metrics: Recognize the limitations of focusing solely on the volume of work being done. Understand that not all lines of code are equally valuable. Outcome: A more nuanced evaluation of productivity and impact.

Summary

Educating Other Executives on Engineering

One of the most valuable roles of an executive is to educate other executives on the workings of engineering, rather than shielding engineering from visibility. A nuanced example cited is the request from a CEO to understand the volume of software, measured in lines of code, being written by different engineers. While such a measure can be misleading, it can serve as a starting point for deeper discussions and understanding of the nuances and limitations of such a metric, rather than dismissing it outright. The goal is to pull people into the details and educate them, rather than forcing them out of the details.

Introduction to the In-depth Show

The In-depth show is a platform that provides tactical advice that founders and startup leaders need to grow their teams, companies, and themselves. The show features weekly conversations with startup leaders that delve deeper into not just what to do, but how to do it.

Guest Introduction - Will Larson

Will Larson, the CTO at Carta, is the guest on the show. He has previously worked as the CTO at Calm, growing the engineering team from 20 to 100 across three different business units. He has also worked at technology companies like Stripe, Uber, Digg, and Yahoo. Larson is a published author of two books on engineering leadership and is well-regarded in the field.

Engineering Strategy Crisis

Will Larson discusses the crisis of engineering strategy, covering why there has never been a consistent approach to engineering strategy and what leaders can do on their teams to measure the success of an engineering organization properly. He also shares insights from his past experiences at Calm, Stripe, and his current role at Carta.

Importance of Documenting Business Decisions

Larson emphasizes that the most important business decisions at a company are often not documented and advises leaders to keep this in mind. This is useful for leaders who are new to a team and need to understand previous decisions or for experienced leaders who want to reassess their strategy documents.

Understanding Context in Engineering Teams

Larson notes that new engineering leaders often struggle when they assume the context from their previous company applies to their new environment. This can lead to problems if the context is not tested and validated in the new environment. In his experience, he has found it beneficial to test ideas and gauge reactions before implementing them, to ensure they are appropriate in the new context.

Testing for Conflict and Assumptions

When entering a new company, it's essential to test for conflict and assumptions rather than just applying what was true at your previous company. It's suggested to have multiple conversations to understand the company's unique reality. This process can reveal the biases and tendencies that people have, which often leads them to continue doing what they're already doing, even if the underlying context or beliefs about it are no longer true.

Understanding Historical Decisions

One challenge in understanding a company's direction is that often, the reasons behind major decisions aren't documented. This requires a sort of 'archaeology' to figure out the rationale behind these decisions. Once understood, it can be determined whether those reasons still hold true. This process is important in deciding whether to continue on a certain path or change course.

Assessing Execution Success

Sometimes, despite the decision making being sound and the problem being real, the execution might not be working. This requires having challenging conversations and looking at examples from other companies for benchmarking. If a task is taking significantly longer than it did at similar companies, it may indicate that something isn't working and needs investigation.

Importance of Benchmarks

Benchmarks are useful for understanding performance, especially for new leaders who lack context. They can also help the team understand how they are doing compared to others. While they may not provide the complete truth, they offer a reasonable guess at whether something is working or not.

Applying Past Experiences to New Situations

A common challenge when moving between companies is testing whether strategies that worked in the past will apply in the new context. This requires double-checking assumptions and updating one's thinking or models. An example given is the transition from Uber to Stripe, where a successful approach at Uber (provisioning automation tooling) was initially rejected at Stripe, highlighting the importance of understanding and adapting to a new company's unique context.

Understanding and Adapting to Different Technological Contexts

Describes an encounter with a colleague named Nelson who was resistant to a proposed technological approach. The speaker initially viewed Nelson as unreasonable, but later understood that their context at Stripe, a company that primarily used the Ruby programming language, was different from the speaker's previous context at Uber. This difference in context led to a realization that the speaker's proposed approach was not aligned with Stripe's technological strategy.

The Importance of Context in Decision Making

Highlights the importance of understanding and respecting the context of others, especially when they have more experience in a particular area. The speaker realized that instead of dismissing Nelson's concerns, it was important to refine their own approach based on the context provided by Nelson.

Undocumented Beliefs in Technology Management

Discusses the existence of undocumented beliefs about technology management in companies like Uber and Stripe. These beliefs influence decisions such as whether to invest in a single programming language or to choose the best language for each problem. The speaker mentions that Uber supports many languages, while Stripe prefers a single tool for all problems.

The Value of Differing Points of View

Emphasizes the value of understanding and representing opposing points of view. The speaker suggests that it's important to understand the opposition's perspective as clearly as your own before moving forward. This approach led to the speaker's realization that Nelson's perspective was a better fit for the problem at hand.

Recognizing the Value of Contextual Knowledge

Explains that executives can often gain buy-in more easily from other executives than from those with the most context around a problem. Those with the most detailed understanding of a problem are the most valuable contributors, as they understand the truth of the situation.

The Danger and Benefit of Pattern Matching

Discusses the concept of pattern matching, which can be both dangerous due to the risk of being wrong, and effective due to its speed and familiarity. The speaker describes an example involving a services migration at a company called Calm, where they had to convince a team to abandon a project that wasn't progressing.

Motivation Behind Technology Adoption

Describes a conversation with a team at Calm who wanted to continue a stalled project because they believed it was the right way to motivate engineers to adopt new technology and do interesting work. The speaker acknowledged the validity of this viewpoint, underscoring the importance of understanding different perspectives.

Company Goals and Employee Motivation

The speaker emphasizes that the goal of a company is not to provide interesting technical work to motivate engineers, as some may believe. Instead, the role of the engineers is to help the company achieve its goals on behalf of the business and its users. This perspective may not align with everyone's values, leading to some employees leaving the company. This is seen as a natural outcome due to value mismatch and doesn't necessarily mean a failure on the part of the company or the individual.

Importance of Defining Company Values

The speaker warns against trying to be a company for everyone. In doing so, a company can become diffuse and confusing, not standing for anything in particular. It's essential to define the company's values and principles, even if it means not being the right fit for everyone. This helps avoid confusion about what the company stands for and attracts those who align with these values.

Role of Written Engineering Principles

The speaker introduces the idea of writing down engineering principles and strategies. Often, employees across different roles feel a lack of strategy due to it not being explicitly written down. The speaker argues that strategies are present everywhere, but are rarely written, making them hard to understand, refine, and implement. Writing strategies down improves clarity, helps onboard new hires, and facilitates discussions around strategy improvements.

Challenges of Unwritten Strategy

Unwritten strategies can be effective but are hard to use and improve. They can lead to different interpretations and confusion, especially when trying to onboard new hires or update the strategy. The speaker advises that writing down strategies, regardless of their current quality, will lead to improvement over time.

Creating a Strategy Document

The speaker discusses two aspects of creating a strategy document: its structure and the process of creating it. They suggest using Richard Rumelt's book, 'Good Strategy Bad Strategy', as a guide for structuring a strategy document. Rumelt's strategy structure consists of three parts: a diagnosis of the current state, guiding policies to interpret the diagnosis, and coherent action to implement the strategy. The speaker emphasizes that a strategy is only real if it is being actively implemented.

Financial Strategy and Guiding Policies

The speaker discusses a hypothetical financial situation of a company with $10 million in the bank, having spent $2.5 million in the previous year with a steady but stagnant revenue stream. The guiding principle in this scenario would be to keep the costs flat year-over-year and maintain the current revenue. This would provide a runway of four years, given the plan to spend $2.5 million annually. Any initiative that would increase this expenditure would be aggressively prevented. The actions to maintain this strategy would include monthly meetings with the finance department to review all expenditures and measures to immediately rectify any increase in spending.

Engineering Strategy: Resource Allocation and Decision Making

Engineering strategy is discussed in the context of two core ideas: resource allocation and decision making. Resource allocation involves deciding how much to invest in infrastructure, developer tooling, and supporting different business lines. This also includes prioritizing projects within those areas. Decision making involves determining how to handle important decisions that come up, such as hiring decisions, requests for additional headcount, and evaluating the adoption of new technologies. The speaker notes that enforcing these strategies and maintaining consistency is often the most challenging aspect.

Impact of Resource Allocation on Engineering Strategy

The speaker notes that many companies have avoided making difficult decisions about resource allocation by simply hiring more people. However, with fixed budgets becoming more of a reality for many companies, this approach is no longer viable. This shift is forcing engineering leadership to make more strategic decisions about resource allocation. This includes deciding which projects to staff and which technologies to adopt, in an environment where resources are more constrained.

Enforcing Strategy, Frameworks, and Policies

The speaker discusses different models that companies use to enforce their strategies. One model involves teams having the freedom to make their own decisions, with problematic issues escalating to executives. This model can enable teams to move quickly and choose their own tools, but it can also lead to late discovery of issues and a lack of concentrated technology investments. Another model involves top-down decision-making by an executive or a community of senior engineers. The speaker suggests that the best approach may depend on the specific circumstances of the company.

Challenges of Different Organizational Structures

The speaker discusses the challenges associated with different organizational structures in technology companies. The Bottoms Up approach, though it allows for a great deal of creativity and freedom, can be very difficult to manage due to the lack of structure. Chief Architects and CTOs, while they can provide a centralized decision-making process, often lack the necessary context for every decision they need to make. Committees, while they bring together a variety of perspectives, can be slow and inefficient in their decision-making process.

The Navigator Approach

The speaker introduces the concept of 'Navigators', a system they've been implementing at Carta. In this system, one person per business unit acts as the CTO for that area, making the final decisions for that area's technology. This person is guided by a written strategy but can make exceptions if they see fit. The speaker believes this approach allows for more informed and context-specific decisions, as the decision-makers are those with the most understanding of their specific area.

Example of Navigator's Role

The speaker provides an example of a Navigator's role in the context of Carta. In one business unit, the Navigator decided to minimize the use of Kafka, a messaging system, as they didn't believe it was the right tool for their specific needs. This decision was made despite other business units deciding to heavily use Kafka. This example illustrates the flexibility and context-specific decision-making that the Navigator system allows for.

Hidden Variables in Team Performance

The speaker discusses the difficulty in understanding the factors that contribute to a team's performance. There are often hidden variables at play that can make it challenging to pinpoint the reasons for a team's success or failure. For example, a team may appear to be performing poorly due to a lack of focus, when the real issue is poor execution. Similarly, a high-performing team may be attributed to certain factors, when in reality it may be due to one individual with excellent intuition or instincts. Understanding these variables is complex and can lead to false conclusions.

Increasing Engineering Velocity

The speaker identifies a common challenge faced by CTOs and heads of engineering: the pressure to increase engineering velocity. However, there's often no clear way to simply increase velocity, as it's not as simple as adding more people to a team. The speaker compares this to the sales department, where it's often believed that adding more salespeople will result in more sales. The speaker believes this is an oversimplification, and that individual talent and abilities play a significant role in both sales and engineering.

Challenges of Measuring Engineering Performance

Engineering is a complex field, and it is difficult to find measures that accurately reflect performance. The speaker explains that teams can vary greatly, with some individuals pulling a lot of weight and others creating conflict. Adding or removing individuals can significantly affect a team's productivity, but it is not always clear how. This uncertainty can be frustrating, especially for leaders who are managing large teams with significant budgets.

Comparing Engineering to Art

The speaker discusses the challenge of predicting the outcomes of engineering projects, likening it to predicting the outcome of creating a piece of art. This unpredictability can be unsettling for those overseeing large budgets, as they may not be reassured by the lack of concrete measures for success.

Importance of Detailed Understanding

The speaker emphasizes the importance of leaders understanding the details of their team's work. This involves identifying the most crucial areas to understand in order to advance the business. This also includes understanding the data, its sources, and how it could potentially be misleading. This detailed understanding helps to identify key individuals and their roles, which can be beneficial for the overall functioning of the team.

The Role of High Performers and Risks Involved

The speaker suggests that increasing productivity often involves reducing the burden on a small number of high performers. However, this strategy carries the risk that if these high performers leave, productivity will drop significantly. This strategy is therefore high-risk and high-reward, and may not work well at larger scales.

Advice for Improving Execution and Addressing CEO Concerns

The speaker advises that improving execution and addressing CEO concerns should ideally be the same problem, but often they are not. To address CEO concerns, they suggest starting to measure something, despite the inevitable imperfections of any measurement system. This helps to educate the CEO and inform their understanding of how things work. The speaker also suggests using metrics not only to show the truth, but also to educate people and inform their mental model of how the truth works.

Informing Mental Models of the Board and CEO

Encourages individuals to focus less on aspects that conflict with their personal mental models and more on elements that can inform the mental models of others, particularly those in higher positions such as the board or CEO. This is seen as a crucial goal when reporting upwards as it helps to shed light on the real substance of the work being done. These measures are dynamic and will change every quarter for the foreseeable future at a company.

Differentiating Between Company Progress and Investment Areas

Highlights two distinct problems: the upward-facing issue of assessing how fast the company is progressing and the downward problem of determining where to invest to accelerate this progress. While certain metrics are useful for understanding where to invest, they do not necessarily reflect the company's rate of progress. Tools like the Accelerate and SPACE frameworks can help identify the right bets and evaluate their effectiveness.

Challenges with Volume-based Metrics

Discusses the industry trend of focusing on the volume of work being done, such as the number and size of pull requests. This approach overlooks individuals providing indirect value, like coaching others or improving the quality of others' PRs. Moreover, there's no direct correlation between code volume and product impact. A request from a CEO to understand the volume of software or lines of code written by different engineers can be challenging and nuanced, as not all lines of code are equally valuable or beneficial.

Building Mental Models through Detailed Metrics

Emphasizes the importance of using detailed metrics to build mental models, rather than rejecting certain metrics outright. Encourages a nuanced understanding of metrics, recognizing their limitations and the need for detailed conversations about their implications. The ultimate goal is to educate other executives on how engineering works, rather than protecting engineering from visibility.

Useful Metrics in Running an Engineering Organization

Begins a discussion on specific metrics that are useful in the context of running an engineering organization, although the content ends before any particular metric is discussed.

Reliability and Service Level Agreement

At Stripe, a reliability Service Level Agreement (SLA) was extended to enterprise customers. This ensured that there were financial consequences if the uptime fell below a certain level, specifically less than four nines (99.99%) of API availability. This commitment to customers was tracked to maintain a high level of service.

Debate on Incident Targets

There was an internal debate at Stripe about setting targets for the number of production incidents. The company classified incidents into different severities such as high, medium, and low. However, setting targets for incidents could lead to counterproductive behavior such as hiding incidents. Instead, a suggestion was made to set targets around the customer impact of incidents. This approach aimed to improve the company's mental model on how to measure reliability.

Importance of Measurements

Measurements, while not perfect, are crucial for refining the company's collective understanding of various areas. Over time, these measurements can be refined. The aim is not to achieve perfection in metrics, but to find something useful that can enhance the company's mental model. Starting with the number of incidents might not be the perfect measure, but it provides a useful platform to refine the company's understanding of how to measure reliability.

Development Process Metrics

Metrics around the development process, such as build velocity, time to run tests, test stability, and the number of large projects shipped, can provide useful insights. While these metrics might be controversial due to the variability in project sizes, they provide a sense of accountability and a way to gauge the frequency and impact of shipping projects. These metrics provide a directional signal that can be drilled into for more detail.

Evolution of Measurement Philosophy

Over time, the speaker's perspective on measurements has evolved. While earlier in their career they were focused on measuring the perfect thing, they now see value in measuring things that give a directional signal. This approach allows for quicker learning and adaptation within the organization.

Engineering Anti-patterns: Micromanagement vs Detail Orientation

An anti-pattern observed in engineering is the confusion between micromanagement and being close to the details. Often, when an engineer transitions into a managerial role, they are advised to step away from the details. However, this can lead to a lack of understanding of the technical context, which is crucial for effective decision-making. Striking a balance between managerial duties and technical understanding is key.

Understanding the Balance Between Micromanagement and Hands-off Leadership

While some people thrive on specific instructions and a clearly defined path, others may find this approach stifling. There's a fine balance between stepping back to allow your team to make decisions and being so hands-off that you're not aware of the details of what your team is doing. Overuse of hands-off advice can lead managers to become detached from the details of their team's work, which can be detrimental to effective management. It's important to know what your team is doing in detail, but without micromanaging them.

The Role of a Manager Beyond Resource Allocation

As managers become more senior, there's a risk of seeing their role simply as a resource allocator, focusing solely on budget allocation and checking the quality of that allocation. While this is an important part of management, it doesn't encompass the entirety of the role. A high-functioning executive needs to understand their domain in detail, know the levers that can transform the business, and be clear on the business outcomes needed and the timeframe for achieving them. Straying too far from the details can lead to becoming a bureaucrat rather than an engaged leader.

The Importance of Diverse Leadership Styles

While micromanagement should not be the default leadership style, there are situations where it can be valuable. For example, when there is no clear path forward or when team members disagree strongly, it can be beneficial for a leader to get into the details and make the decision themselves. This approach involves building your own conviction around what needs to be done. Having a range of leadership styles to choose from can make an executive more impactful.

Avoiding Overapplication of Rules

Applying rules too universally can lead to anti-patterns. For example, while it can be useful to have a guideline that a team should generally consist of one manager to six to eight engineers, applying this rule too rigidly can be problematic. Rules and guidelines should be seen as aids to thinking about an organization and determining ideal ratios, rather than absolute dictates. Rigidity can lead to trouble, so it's important to allow for flexibility and exceptions.

Realities of Policy Exceptions

The discussion emphasizes the importance of adaptability in policies, recognizing that general policies may not always be applicable in specific cases. There needs to be a willingness to consider alternative approaches when necessary. It also stresses the importance of thinking critically about whether a different approach might make more sense in a specific case.

The Pitfalls of Executive Assumptions

The conversation highlights a common mistake made by executives, where they apply their previous company's strategies without considering the unique context of their current company. This approach can lead to difficulties if it doesn't work in the new context, with executives often doubling down or bringing in familiar senior leadership, rather than adapting their own leadership style or approach.

Adaptability of Executives

The segment discusses the importance of adaptability in executive roles. It suggests that it's often easier for an executive to change their leadership style or approach than it is to change the entire organization to fit their style. Executives who are willing to adapt themselves to fit the company are more likely to succeed in the long term.

The Importance of Decision-making in Large Groups

The segment discusses the challenge of consistent decision-making across large groups, such as in organizations with many engineers. It suggests designating 'navigators' or trusted individuals who can interpret strategy consistently for each area. This approach can help maintain alignment and ensure high-quality decisions.

The Role of Judgment in Organizations

The conversation touches on the crucial role of judgment in organizations. It stresses the importance of selecting individuals who are trusted to exercise good judgment. It also discusses the implications of having individuals who are 'load-bearing', or crucial to the organization's functioning, acknowledging that there is no perfect redundancy for good judgment.

Characteristics of an Excellent Engineering Executive

The segment discusses the qualities of an excellent engineering executive. It outlines the challenge of balancing different roles, such as being a member of the executive team, leading the business forward, and making decisions that might not always be favorable for the engineering department.

The Three Dimensions of Engineering Leadership

The speaker discusses three dimensions of engineering leadership: business unit model, engineering management leadership, and technical execution. The business unit model refers to the organizational structure and the voice of engineering in business-related decisions. Engineering management leadership involves creating a structure and implementing processes and policies to run the organization. Technical execution represents the actual engineering work, focusing on technical excellence and implementation. The speaker emphasizes the need to balance these three dimensions to be a top-tier executive, as leaning too heavily on one can lead to inefficiencies and misrepresentation of other crucial aspects.

Role of a Top-Tier Executive

The speaker suggests that top-tier executives should be capable of wearing and switching between different 'hats', representing the three dimensions of engineering leadership. This involves taking into account the business perspective, the management perspective, and the engineering perspective. They also stress the importance of understanding the product and users, maintaining curiosity about how other functions work, and avoiding the arrogance of engineering supremacy.

Evaluating the Performance of a CTO

When evaluating the performance of a CTO, the speaker recommends speaking to the CTO's peers, senior managers, and senior engineers separately, as each group can provide different insights. They outline two scenarios where a CEO might feel a CTO is struggling. The first is when the CTO causes friction with other team members, and the second is when the shipping velocity, or the rate at which work is being completed, does not meet expectations. In the latter case, the speaker suggests drilling into why the shipping velocity is not meeting expectations, which could involve understaffing, technical debt, or issues with other functions.

Addressing Executive Performance Issues

The speaker emphasizes the importance of drilling into details when diagnosing executive performance issues. Rather than seeking a consensus perspective, a CEO needs to form a conviction view on the performance of their executives. This involves gaining multiple perspectives, examining evidence such as product roadmaps, and identifying whether the issue lies in the product or in the executive's performance.

Evolution in Management Style

Reflecting on the changes in his management style over the years, the speaker highlights a shift from protecting team members from reality to exposing them to it. He believes that buffering information can ultimately be detrimental, as it prevents team members from processing and adapting to reality over time. He now prefers to involve his team in the messy details of reality, even if it may initially be disappointing.

Role as a Manager

The speaker discusses how he has become more intentional in his role as a manager. He emphasizes the importance of being a manager for both managers and engineers, ensuring different perspectives are represented. He also highlights the need for regular meetings and offsites with various groups within the organization.

Consistencies in Management Style

Despite the changes in his management style, the speaker maintains a strong belief in communication, particularly written communication. He advocates for inclusive decision-making processes, involving larger groups rather than small ones, and holding individuals accountable for showing up in thoughtful and effective ways. He also supports live conflict resolution among stakeholders.

The Importance of Communication Training

The speaker points out the lack of formal communication training in tech, contrasting it with industries like consulting where communication methods are taught. He suggests that this is a gap in the industry, as effective communication is crucial to organizational success.

Communication Techniques

The speaker discusses the importance of understanding different communication styles when interacting with different groups. For instance, when communicating with executives, it's recommended to start with the conclusion first, as they generally prefer a concise summary rather than a detailed explanation. The speaker advises that if the executives agree with the conclusion, there's no need to present supporting evidence.

Understanding Executive Feedback

The speaker emphasizes the importance of understanding the feedback from executives, even when it seems random or off-topic. Rather than dismissing such feedback or getting into conflict, it's more productive to ask detailed questions and extract the kernel of insight. The speaker also acknowledges that while it may seem unfair that executives can communicate poorly, the onus is on the individuals to understand and extract value from the feedback.

Communicating at the Right Altitude

The speaker introduces the concept of 'flying at the right altitude' when communicating. This refers to providing the right level of context depending on the audience. For instance, when communicating with executives or higher-ups, it's important to provide a higher level of context, while peers or those closer to the day-to-day operations would require more detailed information.

Presentation Techniques for Executives

The presenter highlights the importance of starting a presentation to executives with a broad overview and then gradually zooming in to the specific problem at hand. This approach, referred to as 'zooming in', helps to bring the audience to the same level of understanding as the presenter. The presenter emphasizes the need to do this quickly, ideally within 15-30 seconds, to maintain audience engagement and avoid losing their attention. It's also suggested that the presenter should practice this technique to increase speed and efficiency.

Context Framing and Testing

The presenter suggests testing the presentation with individuals who are closely aligned with the executives, such as an executive assistant, chief of staff, or senior manager. These individuals can provide valuable feedback on the 'altitude' or depth of detail that the presentation should aim for. The presenter emphasizes the importance of this testing phase in calibrating the presentation to the executive's level of understanding.

Common Mistakes in Presentations

The presenter identifies several common mistakes that can derail a presentation. These include coming in with too much conviction and being unwilling to accept feedback, failing to manage attendees and their questions, and not being prepared to deviate from the planned presentation. The presenter advises being open to feedback and being ready to adapt the presentation on the fly if necessary.

Effective Responses and Presentation Delivery

The presenter discusses the concept of 'three Amazon answers', which emphasizes concise, clear responses. He also warns against resisting derailment during the presentation, as it can lead to disengagement. He suggests that since executives see many presentations, they can often consume slides quickly and may already have questions ready, so presenters should be prepared for this.

Managing Presentation Pace

The importance of managing the pace of presentations is discussed. The speaker notes that many individuals tend to control the pacing of a presentation, rather than allowing the listeners to dictate the pace. This can lead to listeners becoming disengaged and distracted. It is suggested that for a high-quality executive meeting, the executive should be allowed to set the pace.

Feedback Processing

The speaker discusses different approaches to processing feedback during meetings. There is a distinction made between feedback that makes no sense and feedback where there is a disagreement within shared context. In the former, it is recommended to ask questions, take time to think it through and revisit later. In the latter, it is valuable to discuss and debate the points live during the meeting, as the shared context is already established.

Revisiting Topics

The topic of revisiting issues or topics during meetings with direct reports is addressed. It is noted that constantly revisiting the same topics can be a negative sign. However, revisiting a topic after a period of time to discuss outcomes and next steps is considered productive. The speaker emphasizes the importance of problem-solving during meetings, especially with senior team members.

Building Fundamentals in Early Career

The speaker discusses the process of building fundamentals in the early stages of an engineering career. Engineering is described as an apprenticeship model where learning occurs through observation, feedback, and repetition. This industry provides opportunities to learn rather than direct teaching.

Engaging with Team Members and Managing Conflict

The speaker discusses the importance of engaging with different team members and creating a space for fruitful conflict. This type of conflict should build trust rather than diminish it. It's essential to work through priorities and create space in the roadmap for testing or quality investment. If there's a need to move faster, this should be addressed by thinking about the fundamentals and guiding people back to it.

Understanding the Term 'Senior'

The speaker questions the meaning of 'senior' in the context of being an engineer, especially when someone can acquire this status in just four years. They highlight that this short timeframe might lead to skipping important skills. They argue that to be an effective senior engineer, you need to see different views on problems and spend time addressing any gaps in knowledge.

Training and Apprenticeship Models

The speaker discusses the use of both training and apprenticeship models in building engineering teams. For senior members, the apprenticeship model works better, as their problems are usually specific and precise. For less experienced members, trainings are beneficial. They also mention that while some people prefer trainings, others don't, hence the need for a mix of both methods.

Effective Trainings

The speaker admits that they don't know which trainings have the biggest impact, as they usually focus on what's going wrong. They mention a training for hiring managers to build conviction about hiring candidates, which addresses a common challenge of team disagreement over candidates. The speaker stresses the importance of managers building their own conviction rather than relying on the average opinion of their team.

Team Composition

The speaker discusses the importance of team composition, particularly for senior managers. They point out that many senior managers initially have many small teams, but these are not resilient. If one person leaves, it can disrupt the entire team. Therefore, it's better to have larger teams with pods within them to deal with potential losses.

Building Consensus

The speaker discusses the importance of building consensus as a part of the job of staff plus engineers. They question why this is not considered a part of the manager's role and delve into how this process works in practice.

Inspirations and Learning from Mistakes

The speaker shares their personal experience, emphasizing how they learn best by doing and from their own mistakes. This has led to introspection and understanding of gaps in their work. They also mention the importance of learning from others who have made similar mistakes, which has shaped their worldviews.

Learning Circle

The speaker talks about their learning circle of CTOs and VPs of engineering that meets bi-weekly for shared learning. This circle has proven to be an invaluable resource for them. They mention that there isn't a single source of information or learning for them, but rather a comprehensive and wide consumption of knowledge from various sources.

Running the Bi-weekly Learning Circle

The speaker explains the structure of their bi-weekly learning circle. They start with a round-the-world segment where each person shares what they are focused on for the week and a topic they'd like to discuss. The group then discusses a variety of topics that have been raised, providing a broad range of insights and learning opportunities.

Longevity of the Learning Circle

The speaker mentions that the learning circle has been running for about three and a half years. They also discuss the dynamic nature of the group, with people coming in and out, and the addition of new members every six months to keep the group fresh and diverse.

Fantastic Set of Topics

The content delves into a truly fantastic set of topics. Though it does not specify the exact topics, the enthusiasm suggests they are of great interest and likely cover a broad range of subjects. The use of 'really really' emphasizes the high quality or intriguing nature of these topics.

FAQs

What is one of the most valuable roles of an executive in engineering? One of the most valuable roles of an executive is to educate other executives on the workings of engineering, rather than shielding engineering from visibility.

What is the In-depth show? The In-depth show is a platform that provides tactical advice that founders and startup leaders need to grow their teams, companies, and themselves. The show features weekly conversations with startup leaders that delve deeper into not just what to do, but how to do it.

Who is Will Larson? Will Larson, the CTO at Carta, is a guest on the show. He has previously worked as the CTO at Calm, growing the engineering team from 20 to 100 across three different business units. He has also worked at technology companies like Stripe, Uber, Digg, and Yahoo. Larson is a published author of two books on engineering leadership and is well-regarded in the field.

What is the crisis of engineering strategy? The crisis of engineering strategy refers to the lack of a consistent approach to engineering strategy and how leaders can measure the success of an engineering organization properly.

Why is it important to document business decisions? Documenting business decisions is important as they are often not kept on record. This is useful for leaders who are new to a team and need to understand previous decisions or for experienced leaders who want to reassess their strategy documents.

Why is it necessary to understand context in engineering teams? Understanding context is necessary because new engineering leaders often struggle when they assume the context from their previous company applies to their new environment. This can lead to problems if the context is not tested and validated in the new environment.

Why is it essential to test for conflict and assumptions when entering a new company? Testing for conflict and assumptions is essential to understand the company's unique reality. This process can reveal the biases and tendencies that people have, which often leads them to continue doing what they're already doing, even if the underlying context or beliefs about it are no longer true.

Why is it important to understand historical decisions? Understanding historical decisions is important because the reasons behind major decisions often aren't documented. Once understood, it can be determined whether those reasons still hold true. This process is important in deciding whether to continue on a certain path or change course.

What do you do when the execution is not working despite sound decision making? When the execution is not working, it requires having challenging conversations and looking at examples from other companies for benchmarking. If a task is taking significantly longer than it did at similar companies, it may indicate that something isn't working and needs investigation.

Why are benchmarks important? Benchmarks are useful for understanding performance, especially for new leaders who lack context. They can also help the team understand how they are doing compared to others. While they may not provide the complete truth, they offer a reasonable guess at whether something is working or not.

What is a common challenge when moving between companies? A common challenge when moving between companies is testing whether strategies that worked in the past will apply in the new context. This requires double-checking assumptions and updating one's thinking or models.

What was the speaker's realization about different technological contexts at Stripe and Uber? The speaker realized that their proposed technological approach, which was aligned with their previous experience at Uber, was not suitable for Stripe due to the company's unique context and primary use of the Ruby programming language.

What is the importance of context in decision making according to the speaker? The speaker highlights the importance of understanding and respecting the context of others, especially when they have more experience in a particular area. Instead of dismissing others' concerns, it's important to refine one's own approach based on the context provided by others.

What are undocumented beliefs in technology management? Undocumented beliefs in technology management are the unwritten convictions that influence decisions such as whether to invest in a single programming language or to choose the best language for each problem. These beliefs exist in companies like Uber and Stripe.

What is the value of differing points of view according to the speaker? The speaker emphasizes the value of understanding and representing opposing points of view. It's important to understand the opposition's perspective as clearly as your own before moving forward.

What is the value of contextual knowledge in problem-solving? Contextual knowledge is valuable because those with the most detailed understanding of a problem can provide the most accurate solutions. They understand the truth of the situation and can often gain buy-in more easily from other executives.

What is pattern matching and what are its pros and cons? Pattern matching is a concept that can be dangerous due to the risk of being wrong, but effective due to its speed and familiarity. It involves using past experiences to quickly identify and respond to similar situations.

Why is understanding different perspectives important in technology adoption? Understanding different perspectives is important in technology adoption because it allows for the recognition of valid viewpoints that may differ from your own. This can lead to better decision-making and more successful implementation of new technologies.

What is the role of engineers in a company according to the speaker? According to the speaker, the role of engineers is not just to be motivated by interesting technical work, but to help the company achieve its goals on behalf of the business and its users.

Why is it important to define a company's values and principles? Defining a company's values and principles is important because it helps avoid confusion about what the company stands for and attracts those who align with these values. It prevents the company from becoming diffuse and not standing for anything in particular.

What is the role of written engineering principles according to the speaker? Written engineering principles and strategies can improve clarity, help onboard new hires, and facilitate discussions around strategy improvements. They prevent confusion and different interpretations that can arise from unwritten strategies.

What are the challenges of unwritten strategy? Unwritten strategies can lead to different interpretations and confusion, especially when trying to onboard new hires or update the strategy. They can be hard to use and improve.

What is the benefit of creating a strategy document? Creating a strategy document, regardless of the current quality of the strategies, can improve clarity and facilitate discussions around strategy improvements. It can also help in onboarding new hires and updating the strategy.

What are the two aspects of creating a strategy document? The two aspects of creating a strategy document are its structure and the process of creating it.

What is the structure of a strategy document according to Richard Rumelt? Rumelt's strategy structure consists of three parts: a diagnosis of the current state, guiding policies to interpret the diagnosis, and coherent action to implement the strategy.

What does the speaker suggest about implementing a strategy? The speaker emphasizes that a strategy is only real if it is being actively implemented.

What are the two core ideas of engineering strategy? The two core ideas of engineering strategy are resource allocation and decision making.

Why is enforcing strategies and maintaining consistency often the most challenging aspect? Enforcing these strategies and maintaining consistency is often the most challenging aspect because it involves making difficult decisions about resource allocation and decision making.

What is the impact of resource allocation on engineering strategy? The impact of resource allocation on engineering strategy is that it forces engineering leadership to make more strategic decisions about which projects to staff and which technologies to adopt, especially in an environment where resources are more constrained.

What are the challenges associated with different organizational structures in technology companies? The challenges associated with different organizational structures in technology companies include managing a lack of structure in a Bottoms Up approach, centralized decision-making processes lacking necessary context, and slow and inefficient decision-making processes in committees.

What is the 'Navigator' approach? The 'Navigator' approach is a system where one person per business unit acts as the CTO for that area, making the final decisions for that area's technology. This person is guided by a written strategy but can make exceptions if they see fit.

What is an example of a Navigator's role? An example of a Navigator's role is when the Navigator for one business unit decided to minimize the use of Kafka, a messaging system, as they didn't believe it was the right tool for their specific needs.

What are the hidden variables in team performance? The hidden variables in team performance are the factors that contribute to a team's performance that can make it challenging to pinpoint the reasons for a team's success or failure.

What is a common challenge faced by CTOs and heads of engineering? A common challenge faced by CTOs and heads of engineering is the pressure to increase engineering velocity. However, it is often not clear how to simply increase velocity, as it's not as simple as adding more people to a team.

Why is it difficult to measure engineering performance? Engineering is a complex field and it is difficult to find measures that accurately reflect performance. Teams can vary greatly, with some individuals pulling a lot of weight and others creating conflict. Adding or removing individuals can significantly affect a team's productivity, but it is not always clear how.

What is the challenge of predicting the outcomes of engineering projects? Predicting the outcomes of engineering projects can be challenging, as it can be likened to predicting the outcome of creating a piece of art. This unpredictability can be unsettling for those overseeing large budgets, as they may not be reassured by the lack of concrete measures for success.

What is the importance of leaders understanding the details of their team's work? Understanding the details of their team's work is crucial for leaders. This involves identifying the most crucial areas to understand in order to advance the business. It also includes understanding the data, its sources, and how it could potentially be misleading. This detailed understanding helps to identify key individuals and their roles, which can be beneficial for the overall functioning of the team.

What is the role of high performers in increasing productivity and what are the risks involved? Increasing productivity often involves reducing the burden on a small number of high performers. However, this strategy carries the risk that if these high performers leave, productivity will drop significantly. This strategy is therefore high-risk and high-reward, and may not work well at larger scales.

How can you improve execution and address CEO concerns? Improving execution and addressing CEO concerns should ideally be the same problem, but often they are not. To address CEO concerns, it's suggested to start measuring something, despite the inevitable imperfections of any measurement system. This helps to educate the CEO and inform their understanding of how things work.

What is the aim when reporting upwards? When reporting upwards, the aim should be to inform the mental models of others, particularly those in higher positions such as the board or CEO. This helps to shed light on the real substance of the work being done.

What are the two distinct problems when assessing a company's progress and investment areas? The two distinct problems are the upward-facing issue of assessing how fast the company is progressing and the downward problem of determining where to invest to accelerate this progress.

What is the challenge with volume-based metrics in the industry? The industry trend of focusing on the volume of work being done, such as the number and size of pull requests, overlooks individuals providing indirect value, like coaching others or improving the quality of others' PRs. Moreover, there's no direct correlation between code volume and product impact.

What is the importance of using detailed metrics? Using detailed metrics is important as it helps in building mental models, educating people, and informing their understanding of how things work.

What is the ultimate goal of understanding metrics in engineering? The ultimate goal is to educate other executives on how engineering works, rather than protecting engineering from visibility.

What was the purpose of the reliability Service Level Agreement (SLA) at Stripe? The reliability SLA at Stripe was extended to enterprise customers to ensure that there were financial consequences if the uptime fell below a certain level, specifically less than four nines (99.99%) of API availability. This commitment to customers was tracked to maintain a high level of service.

What was the internal debate at Stripe about production incidents? The internal debate at Stripe was about setting targets for the number of production incidents. However, setting targets for incidents could lead to counterproductive behavior such as hiding incidents. Instead, a suggestion was made to set targets around the customer impact of incidents.

What is the importance of measurements in a company? Measurements, while not perfect, are crucial for refining the company's collective understanding of various areas. Over time, these measurements can be refined. The aim is not to achieve perfection in metrics, but to find something useful that can enhance the company's mental model.

What can metrics around the development process provide? Metrics around the development process, such as build velocity, time to run tests, test stability, and the number of large projects shipped, can provide useful insights. They provide a sense of accountability and a way to gauge the frequency and impact of shipping projects.

What is the difference between micromanagement and being close to details in engineering? Micromanagement is often confused with being close to the details. When an engineer transitions into a managerial role, they are advised to step away from the details. However, this can lead to a lack of understanding of the technical context, which is crucial for effective decision-making. Striking a balance between managerial duties and technical understanding is key.

What is the balance between micromanagement and hands-off leadership? There's a fine balance between stepping back to allow your team to make decisions and being so hands-off that you're not aware of the details of what your team is doing. Overuse of hands-off advice can lead managers to become detached from the details of their team's work, which can be detrimental to effective management.

What is the role of a manager beyond resource allocation? A high-functioning executive needs to understand their domain in detail, know the levers that can transform the business, and be clear on the business outcomes needed and the timeframe for achieving them. Straying too far from the details can lead to becoming a bureaucrat rather than an engaged leader.

Why is diverse leadership style important? While micromanagement should not be the default leadership style, there are situations where it can be valuable. For example, when there is no clear path forward or when team members disagree strongly, it can be beneficial for a leader to get involved.

What are the problems with overapplying rules in an organization? Applying rules too universally can lead to anti-patterns. Rigidity can lead to trouble, so it's important to allow for flexibility and exceptions.

Why is adaptability important in policies? General policies may not always be applicable in specific cases. There needs to be a willingness to consider alternative approaches when necessary.

What is a common mistake made by executives? A common mistake made by executives is applying their previous company's strategies without considering the unique context of their current company.

Why is adaptability important in executive roles? It's often easier for an executive to change their leadership style or approach than it is to change the entire organization to fit their style. Executives who are willing to adapt themselves to fit the company are more likely to succeed in the long term.

What is the challenge of decision-making in large groups? The challenge of consistent decision-making across large groups can be addressed by designating 'navigators' or trusted individuals who can interpret strategy consistently for each area.

What role does judgment play in organizations? Judgment plays a crucial role in organizations. It is important to select individuals who are trusted to exercise good judgment.

What are the qualities of an excellent engineering executive? An excellent engineering executive balances different roles, such as being a member of the executive team, leading the business forward, and making decisions that might not always be favorable for the engineering department.

What are the three dimensions of engineering leadership? The three dimensions of engineering leadership are business unit model, engineering management leadership, and technical execution.

What is the role of a top-tier executive? Top-tier executives should be capable of wearing and switching between different 'hats', representing the three dimensions of engineering leadership.

How to evaluate the performance of a CTO? When evaluating the performance of a CTO, it is recommended to speak to the CTO's peers, senior managers, and senior engineers separately, as each group can provide different insights.

How can drilling into details help when diagnosing executive performance issues? Drilling into details can help a CEO form a conviction view on the performance of their executives. This involves gaining multiple perspectives, examining evidence such as product roadmaps, and identifying whether the issue lies in the product or in the executive's performance.

What is the speaker's current management style? The speaker prefers to expose team members to the reality of situations rather than buffering information. He also emphasizes the importance of being a manager for both managers and engineers, ensuring different perspectives are represented.

What are the consistencies in the speaker's management style? Despite changes in his management style, the speaker maintains a strong belief in communication, particularly written communication. He advocates for inclusive decision-making processes, involving larger groups rather than small ones, and holding individuals accountable for showing up in thoughtful and effective ways.

What is the importance of understanding executive feedback? Understanding the feedback from executives is important, even when it seems random or off-topic. Rather than dismissing such feedback, it's more productive to ask detailed questions and extract the kernel of insight.

What does 'flying at the right altitude' mean in communication? 'Flying at the right altitude' refers to providing the right level of context depending on the audience. For instance, when communicating with executives or higher-ups, it's important to provide a higher level of context, while peers or those closer to the day-to-day operations would require more detailed information.

What is the 'zooming in' technique in presentations for executives? 'Zooming in' is an approach where a presentation starts with a broad overview and then gradually zooms in to the specific problem at hand. This helps to bring the audience to the same level of understanding as the presenter.

Why is testing a presentation important? Testing the presentation with individuals who are closely aligned with the executives can provide valuable feedback on the 'altitude' or depth of detail that the presentation should aim for. This testing phase is important in calibrating the presentation to the executive's level of understanding.

What are some common mistakes that can derail a presentation? Common mistakes include coming in with too much conviction and being unwilling to accept feedback, failing to manage attendees and their questions, and not being prepared to deviate from the planned presentation.

What are 'three Amazon answers'? The 'three Amazon answers' concept emphasizes concise, clear responses during a presentation.

How can the pace of presentations be managed effectively? The pace of presentations can be managed effectively by allowing the listeners to dictate the pace rather than the presenter controlling it.

What are the suggested approaches to processing feedback during meetings? If feedback makes no sense, it is recommended to ask questions, take time to think it through and revisit later. If there is disagreement within shared context, it is valuable to discuss and debate the points live during the meeting.

What is the importance of revisiting topics during meetings? Revisiting a topic after a period of time to discuss outcomes and next steps is considered productive. However, constantly revisiting the same topics can be a negative sign.

What is the process of building fundamentals in the early stages of an engineering career? Engineering is described as an apprenticeship model where learning occurs through observation, feedback, and repetition.

How can team members effectively manage conflict? Engaging with different team members and creating a space for fruitful conflict that builds trust rather than diminishes it is important. It's also essential to work through priorities and create space in the roadmap for testing or quality investment.

What is the meaning of 'senior' in the context of being an engineer? The term 'senior' may be misleading as it may be attained in a short timeframe and lead to skipping important skills. An effective senior engineer should see different views on problems and spend time addressing any gaps in knowledge.

What are the benefits of training and apprenticeship models in building engineering teams? For senior members, the apprenticeship model works better, as their problems are usually specific and precise. For less experienced members, trainings are beneficial.

What is the importance of managers building their own conviction in hiring candidates? Managers should build their own conviction about hiring candidates rather than relying on the average opinion of their team. This addresses a common challenge of team disagreement over candidates.

What is the importance of team composition? Team composition is particularly important for senior managers. It's better to have larger teams with pods within them to deal with potential losses, as many small teams are not resilient.

What is the importance of building consensus? Building consensus is important as it ensures everyone is on the same page and can help prevent conflicts and misunderstandings.

What is the role of building consensus in the job of staff plus engineers? Building consensus is a part of the job of staff plus engineers. They are often required to work in collaboration with others to reach a common understanding or agreement.

How does the speaker learn best? The speaker learns best by doing and from their own mistakes. They also learn from others who have made similar mistakes.

What is the speaker's learning circle? The speaker's learning circle is a group of CTOs and VPs of engineering that meets bi-weekly for shared learning. It's a comprehensive and wide consumption of knowledge from various sources.

How is the bi-weekly learning circle run? The bi-weekly learning circle starts with a round-the-world segment where each person shares what they are focused on for the week and a topic they'd like to discuss. The group then discusses a variety of topics that have been raised.

How long has the learning circle been running? The learning circle has been running for about three and a half years.

What is the dynamic nature of the group? The group is dynamic with people coming in and out, and new members being added every six months to keep the group fresh and diverse.

What kind of topics does the content delve into? The content delves into a truly fantastic set of topics. Though it does not specify the exact topics, the enthusiasm suggests they are of great interest and likely cover a broad range of subjects.

Glossary

Applying Past Experiences to New Situations: A common challenge when moving between companies is testing whether strategies that worked in the past will apply in the new context. This requires double-checking assumptions and updating one's thinking or models.

Assessing Execution Success: Sometimes, despite the decision making being sound and the problem being real, the execution might not be working. This requires having challenging conversations and looking at examples from other companies for benchmarking.

Benchmarks: Benchmarks are useful for understanding performance, especially for new leaders who lack context. They can also help the team understand how they are doing compared to others.

Documenting Business Decisions: The most important business decisions at a company are often not documented. This is useful for leaders who are new to a team and need to understand previous decisions or for experienced leaders who want to reassess their strategy documents.

Educating Other Executives on Engineering: One of the most valuable roles of an executive is to educate other executives on the workings of engineering, rather than shielding engineering from visibility.

Engineering Strategy Crisis: Will Larson discusses the crisis of engineering strategy, covering why there has never been a consistent approach to engineering strategy and what leaders can do on their teams to measure the success of an engineering organization properly.

Guest Introduction - Will Larson: Will Larson, the CTO at Carta, is the guest on the show. He has previously worked as the CTO at Calm, growing the engineering team from 20 to 100 across three different business units.

In-depth Show: The In-depth show is a platform that provides tactical advice that founders and startup leaders need to grow their teams, companies, and themselves.

Testing for Conflict and Assumptions: When entering a new company, it's essential to test for conflict and assumptions rather than just applying what was true at your previous company.

Understanding Context in Engineering Teams: New engineering leaders often struggle when they assume the context from their previous company applies to their new environment. This can lead to problems if the context is not tested and validated in the new environment.

Understanding Historical Decisions: One challenge in understanding a company's direction is that often, the reasons behind major decisions aren't documented. This requires a sort of 'archaeology' to figure out the rationale behind these decisions.

Challenges of Unwritten Strategy: Unwritten strategies can be effective but are hard to use and improve. They can lead to different interpretations and confusion, especially when trying to onboard new hires or update the strategy.

Company Goals and Employee Motivation: The role of engineers is to help the company achieve its goals on behalf of the business and its users. This perspective may not align with everyone's values, leading to some employees leaving the company.

Creating a Strategy Document: Writing down strategies, regardless of their current quality, will lead to improvement over time.

Importance of Defining Company Values: It's essential to define the company's values and principles, even if it means not being the right fit for everyone. This helps avoid confusion about what the company stands for and attracts those who align with these values.

Motivation Behind Technology Adoption: Understanding different perspectives is crucial when motivating engineers to adopt new technology and do interesting work.

Recognizing the Value of Contextual Knowledge: Those with the most detailed understanding of a problem are the most valuable contributors, as they understand the truth of the situation.

Role of Written Engineering Principles: Writing down engineering principles and strategies improves clarity, helps onboard new hires, and facilitates discussions around strategy improvements.

The Danger and Benefit of Pattern Matching: Pattern matching can be both dangerous due to the risk of being wrong, and effective due to its speed and familiarity.

The Importance of Context in Decision Making: Understanding and respecting the context of others, especially when they have more experience in a particular area, is crucial in decision making.

The Value of Differing Points of View: Understanding and representing opposing points of view is valuable. It's important to understand the opposition's perspective as clearly as your own before moving forward.

Understanding and Adapting to Different Technological Contexts: Understanding the context of different technologies is important when proposing a technological approach. A difference in context can lead to a realization that the proposed approach is not aligned with the company's technological strategy.

Undocumented Beliefs in Technology Management: Undocumented beliefs about technology management can influence decisions such as whether to invest in a single programming language or to choose the best language for each problem.

Strategy Document: A document that outlines a business's strategic plan. It is composed of three parts: a diagnosis of the current state, guiding policies to interpret the diagnosis, and coherent action to implement the strategy.

Financial Strategy: A plan of action that allows a business to make the best use of its financial resources to achieve its objectives. It may include keeping costs flat year-over-year and maintaining the current revenue.

Engineering Strategy: A plan that guides resource allocation and decision making in an engineering context. It involves deciding how much to invest in infrastructure, developer tooling, and supporting different business lines, as well as how to handle important decisions.

Resource Allocation: The process of deciding how to distribute resources in an organization. In an engineering context, it involves deciding which projects to staff and which technologies to adopt, especially when resources are constrained.

Enforcing Strategy: The process of ensuring that a business's strategy is being adhered to. This can involve different models, such as allowing teams to make their own decisions or implementing top-down decision-making.

Organizational Structures: The ways in which an organization arranges its people and jobs so that its work can be performed and its goals can be met. Different structures have different advantages and challenges.

Navigator Approach: A system where one person per business unit acts as the CTO for that area, making the final decisions for that area's technology. This person is guided by a written strategy but can make exceptions if they see fit.

Hidden Variables in Team Performance: Factors that are not readily apparent but can significantly impact a team's performance. These can make it challenging to pinpoint the reasons for a team's success or failure.

Increasing Engineering Velocity: A common challenge faced by CTOs and heads of engineering, which is the pressure to increase the rate at which projects are completed. It's not as simple as adding more people to a team, as individual talent and abilities play a significant role.

Challenges of Measuring Engineering Performance: The difficulty in finding measures that accurately reflect performance in engineering. Teams can vary greatly, with some individuals pulling a lot of weight and others creating conflict. Adding or removing individuals can significantly affect a team's productivity.

Comparing Engineering to Art: The challenge of predicting the outcomes of engineering projects, likened to predicting the outcome of creating a piece of art. This unpredictability can be unsettling for those overseeing large budgets.

Importance of Detailed Understanding: The need for leaders to understand the details of their team's work. This involves identifying the most crucial areas to understand to advance the business, understanding the data, its sources, and how it could potentially be misleading.

The Role of High Performers and Risks Involved: The concept that increasing productivity often involves reducing the burden on a small number of high performers. However, this strategy carries the risk that if these high performers leave, productivity will drop significantly.

Advice for Improving Execution and Addressing CEO Concerns: The suggestion that improving execution and addressing CEO concerns should ideally be the same problem. To address CEO concerns, it is advised to start measuring something, despite the inevitable imperfections of any measurement system.

Informing Mental Models of the Board and CEO: The encouragement for individuals to focus less on aspects that conflict with their personal mental models and more on elements that can inform the mental models of others, particularly those in higher positions such as the board or CEO.

Differentiating Between Company Progress and Investment Areas: The identification of two distinct problems: the upward-facing issue of assessing how fast the company is progressing and the downward problem of determining where to invest to accelerate this progress.

Challenges with Volume-based Metrics: The industry trend of focusing on the volume of work being done, such as the number and size of pull requests. This approach overlooks individuals providing indirect value.

Building Mental Models through Detailed Metrics: The emphasis on the importance of using detailed metrics to build mental models and understandings.

Debate on Incident Targets: An internal discussion about setting goals for the number of production incidents. The idea is to improve the company's mental model on how to measure reliability.

Development Process Metrics: Metrics related to the development process, such as build velocity, time to run tests, test stability, and the number of large projects shipped, which provide useful insights.

Engineering Anti-patterns: Micromanagement vs Detail Orientation: A common mistake in engineering where there is confusion between micromanagement and being close to the details. Striking a balance between managerial duties and technical understanding is key.

Evolution of Measurement Philosophy: A shift in perspective on measurements. The importance lies in measuring things that give a directional signal, allowing for quicker learning and adaptation within the organization.

Importance of Measurements: Measurements are crucial for refining the company's collective understanding of various areas. The aim is to find something useful that can enhance the company's mental model.

Reliability and Service Level Agreement: A reliability Service Level Agreement (SLA) ensures that there are financial consequences if the uptime falls below a certain level, specifically less than four nines (99.99%) of API availability.

The Importance of Diverse Leadership Styles: While micromanagement should not be the default leadership style, there are situations where it can be valuable, for example, when there is no clear path forward or when team members disagree strongly.

The Role of a Manager Beyond Resource Allocation: A manager's role is not just about resource allocation. A high-functioning executive needs to understand their domain in detail, know the levers that can transform the business, and be clear on the business outcomes needed and the timeframe for achieving them.

Understanding the Balance Between Micromanagement and Hands-off Leadership: There's a fine balance between allowing your team to make decisions and being so hands-off that you're not aware of the details of what your team is doing. It's important to know what your team is doing in detail, but without micromanaging them.

Useful Metrics in Running an Engineering Organization: A discussion on specific metrics that are useful in the context of running an engineering organization.

Adaptability of Executives: It's often easier for an executive to change their leadership style or approach than it is to change the entire organization to fit their style. Executives who are willing to adapt themselves to fit the company are more likely to succeed in the long term.

Avoiding Overapplication of Rules: Applying rules too universally can lead to anti-patterns. Rules and guidelines should be seen as aids to thinking about an organization and determining ideal ratios, rather than absolute dictates. Rigidity can lead to trouble, so it's important to allow for flexibility and exceptions.

Characteristics of an Excellent Engineering Executive: An excellent engineering executive faces the challenge of balancing different roles, such as being a member of the executive team, leading the business forward, and making decisions that might not always be favorable for the engineering department.

Evaluating the Performance of a CTO: When evaluating the performance of a CTO, it is recommended to speak to the CTO's peers, senior managers, and senior engineers separately, as each group can provide different insights.

Importance of Decision-making in Large Groups: Consistent decision-making across large groups, such as in organizations with many engineers, is challenging. Designating 'navigators' or trusted individuals who can interpret strategy consistently for each area can help maintain alignment and ensure high-quality decisions.

Pitfalls of Executive Assumptions: A common mistake made by executives is applying their previous company's strategies without considering the unique context of their current company. This approach can lead to difficulties if it doesn't work in the new context.

Realities of Policy Exceptions: General policies may not always be applicable in specific cases. There needs to be a willingness to consider alternative approaches when necessary. It also stresses the importance of thinking critically about whether a different approach might make more sense in a specific case.

Role of Judgment in Organizations: Judgment plays a crucial role in organizations. It is important to select individuals who are trusted to exercise good judgment. The implications of having individuals who are 'load-bearing', or crucial to the organization's functioning, acknowledges that there is no perfect redundancy for good judgment.

Role of a Top-Tier Executive: Top-tier executives should be capable of wearing and switching between different 'hats', representing the three dimensions of engineering leadership. This involves taking into account the business perspective, the management perspective, and the engineering perspective.

The Three Dimensions of Engineering Leadership: The three dimensions of engineering leadership are: business unit model, engineering management leadership, and technical execution. Balancing these three dimensions is necessary to be a top-tier executive, as leaning too heavily on one can lead to inefficiencies and misrepresentation of other crucial aspects.

Addressing Executive Performance Issues: The process of diagnosing executive performance issues by drilling into details. This involves gaining multiple perspectives, examining evidence such as product roadmaps, and identifying whether the issue lies in the product or in the executive's performance.

Communication Techniques: The importance of understanding different communication styles when interacting with different groups. For instance, when communicating with executives, it's recommended to start with the conclusion first, as they generally prefer a concise summary rather than a detailed explanation.

Common Mistakes in Presentations: The general errors that are often made when delivering presentations. These may vary depending on the context and audience.

Communicating at the Right Altitude: The concept of providing the right level of context depending on the audience. For instance, when communicating with executives or higher-ups, it's important to provide a higher level of context, while peers or those closer to the day-to-day operations would require more detailed information.

Consistencies in Management Style: The aspects of a manager's approach that remain constant despite changes in other areas. These often include communication, particularly written communication, and inclusive decision-making processes.

Context Framing and Testing: The process of testing a presentation with individuals who are closely aligned with the executives, such as an executive assistant, chief of staff, or senior manager. These individuals can provide valuable feedback on the 'altitude' or depth of detail that the presentation should aim for.

Evolution in Management Style: The changes in a manager's approach over time. This may involve shifts from protecting team members from reality to exposing them to it.

Importance of Communication Training: The necessity of formal communication training in tech, contrasting it with industries like consulting where communication methods are taught.

Presentation Techniques for Executives: The approach of starting a presentation to executives with a broad overview and then gradually zooming in to the specific problem at hand. This helps to bring the audience to the same level of understanding as the presenter.

Role as a Manager: The responsibilities and approach taken by a manager. This includes being a manager for both managers and engineers, ensuring different perspectives are represented, and the need for regular meetings and offsites with various groups within the organization.

Understanding Executive Feedback: The importance of understanding the feedback from executives, even when it seems random or off-topic. It's more productive to ask detailed questions and extract the kernel of insight from the feedback.

Building Consensus: The process of achieving agreement within a group. This often involves discussion, persuasion, and compromise.

Building Fundamentals in Early Career: The process of gaining essential skills and knowledge in the early stages of a career. In engineering, this often involves learning through observation, feedback, and repetition.

Effective Responses and Presentation Delivery: The practice of providing concise, clear responses and managing the flow of a presentation to maintain engagement.

Effective Trainings: Trainings that have a positive impact on the participants. The effectiveness of a training can depend on various factors including content, delivery, and relevance to the participants' needs.

Engaging with Team Members and Managing Conflict: The process of interacting effectively with team members and handling disagreements in a way that builds rather than diminishes trust.

Feedback Processing: The method of interpreting and responding to feedback. This can involve asking questions, considering the feedback, and discussing points of disagreement.

Managing Presentation Pace: The practice of controlling the speed of a presentation to maintain listener engagement. This can involve allowing the listeners to dictate the pace.

Revisiting Topics: The practice of discussing issues or topics again after some time to evaluate outcomes and plan next steps.

Team Composition: The makeup of a team in terms of roles, skills, and numbers. A well-composed team can be more resilient and effective.

Training and Apprenticeship Models: Two methods of learning and development. Training typically involves direct teaching, while apprenticeship involves learning through observation and practice.

Understanding the Term 'Senior': In the context of engineering, 'senior' often refers to someone with advanced skills and experience. However, the exact meaning can vary.

Building Consensus: It is a part of a job of staff plus engineers where they agree on a certain decision or plan. This process is often questioned as to why it's not part of the manager's role and how it works in practice.

Inspirations and Learning from Mistakes: This refers to the process of gaining insight and wisdom from personal experiences, especially from mistakes made. It emphasizes the importance of learning not only from personal mistakes but also from the mistakes of others.

Learning Circle: It is a group of individuals, in this case, CTOs and VPs of engineering, who meet regularly for shared learning. The group is an invaluable resource for learning and knowledge consumption from a variety of sources.

Running the Bi-weekly Learning Circle: This is the structure and process of conducting the bi-weekly learning circle. It involves sharing individual focuses and topics for discussion, followed by a group discussion on a range of topics.

Longevity of the Learning Circle: This refers to the duration for which the learning circle has been running. It also refers to the dynamic nature of the group, with changing members to maintain freshness and diversity.

Fantastic Set of Topics: This term suggests a collection of discussion topics that are of high interest and cover a wide range of subjects. It emphasizes the quality and intriguing nature of these topics.

Resources

Book - Engineering Leadership by Will Larson: Will Larson is a published author of two books on engineering leadership. He has shared his experiences and insights from his roles at technology companies like Stripe, Uber, Digg, and Yahoo, and his current role as the CTO at Carta.

Book - Good Strategy Bad Strategy: This book by Richard Rumelt provides a guide for structuring a strategy document. It discusses the three parts of a strategy: a diagnosis of the current state, guiding policies to interpret the diagnosis, and coherent action to implement the strategy.