Transcript
Shane Hastie: Good day, folks. This is Shane Hastie for the InfoQ, Engineering Culture Podcast. Today I’m sitting down with David Grizzanti. David, I believe is a principal engineer at the New York Times. Am I correct? Still there?
David Grizzanti: That is correct.
Shane Hastie: Welcome. Thank you very much for taking time to talk to us today.
David Grizzanti: Thanks for having me.
Introductions [01:09]
So yeah, as Shane mentioned, I’m a principal engineer at the New York Times located in Philadelphia.
Shane Hastie: So what does it take to be a principal engineer in the New York Times?
David Grizzanti: I think that the title sometimes gives people certain feelings about your job and I think I’ve learned more recently after switching roles after a while that not to get too attached to titles. Each company has their own career ladder. Where I came from, my previous role at Comcast I was a distinguished engineer, and I think originally when I was looking for new roles I was like, I don’t want to give up this title. It sounds so important. But I think I found that the Comcast titles and the New York Times titles didn’t really align necessarily and it’s similar in other companies.
But I think when I was searching for my next role, one of the things that I found or was leading towards was I wanted a position that would give me a little bit of creative freedom for what I worked on, but also a certain level of impact in the organization. And I think that principle level gives you a good feel for that you’re still an individual contributor, you still get to do engineering work, but you still have some influence over technical direction for the company or at least at your organizational level. So I found a good fit there when I joined the Times.
Shane Hastie: And what was your background? How did you get to this place?
David Grizzanti: I’ve always been in what I think is now considered the platform engineering space or maybe a few years ago would be considered DevOps. I started out my career in 2006 at a company called Vonage, people have heard of that, IP telephony company, and I was in a support services role doing software development for all the tools that they ran internally that were built in-house. And this was pre the days of having a lot of SaaS software.
And after working there I went to another service provider company doing cloud development, so infrastructure as a service for medium-sized businesses. And then I went to the Comcast after that and had a similar platform as a service role. They’re doing distributed systems development on tools that you would probably get from a cloud provider now. So like messaging as a service, database as a service, those sorts of things. Things that needed to be built in-house at the time because they weren’t readily available as a SaaS service. And I think all of that prepared me pretty well to enter this developer productivity slash platform engineering space that I’m in now at the Times.
Shane Hastie: I came across you through giving a couple of talks at QCon. QCon, obviously part of the C4Media group, part of in InfoQ. Let’s tackle the latest one from San Francisco, organization resilience was the track and you brought an interesting perspective on it, documentation for organization resilience, but isn’t documentation the thing we love to hate?
Documentation as a facilitator for organizational resilience [03:55]
David Grizzanti: Definitely, and I think that what documentation often suffers from is having too much of it can sometimes be as bad as having not enough at all. And the idea I was curating with that talk and in that track was how can good writing culture and documentation help your organization be more resilient, whether it be because you have attrition, people leaving, people getting hired, and they’re constantly having to ask the same questions over and over again.
One of the examples I gave in the talk was something I see pretty commonly day to day, which is folks will come to our channel on our internal messaging tool like Slack and ask something that’s already documented. And why is that? Is that because they can’t find the documentation? Is because they would prefer to ask and we see this problem over and over, not that it’s a problem, but it’s just like it’s a time sink for people to constantly monitoring the channel and answering questions.
So this idea of reading and writing culture has been interesting to me. I’ve seen Google and other companies put out these technical writing courses that people can take to learn to craft writing in a way that makes it more readable and condense it down into something that’s less verbose.
The other thing that I’ve been interested in is just discoverability of documentation because most companies, every company I’ve been at has lots of different tools, whether it be like Word documents, Google Docs, things in PowerPoint, things that are drawn as diagrams, things in GitHub, things as READMEs, as make doc sites, and it’s really hard to search all of that in a way that’s easy, even if they were all Google documents for instance, it’s so hard to find things.
I gave the example of 10 years ago, Google used to sell this appliance that you could put in your data center that would basically make all of your internal documentation like its own Google searchable thing, but they stopped selling that and there’s not really one perfect solution to it. So I was going through some tools that might help with that, but I think it’s something that is… Not that it’s getting worse, but that drift of documentation or out of date stuff or not existing at all just makes onboarding really hard at new companies, and I haven’t seen anybody be really great at it in my experience.
Shane Hastie: What are some practical things folks can do to move towards better documentation practices?
Tips for better documentation practices [06:07]
David Grizzanti: One of the things I’ve been advocating for is to try and do more work asynchronously by writing documentations about the work you’re doing ahead of time, whether it be like RFCs, ADRs, architecture, decision records, all sorts of things, so that you have documented decisions written down and all kind of centralized in one place. Maybe you have an ADR repository, RFC repository.
There’s some companies that have a culture around this that have seen that are good sort of… I know HashiCorp is one that I’ve seen has a good culture around it, and that’s a culture change, or like I think it’s easier for people to say, okay, we’re going to work on this feature or I’m going to work on this story and be light on what gets written down from an ideation perspective. Maybe they’ll draw a few diagrams, but it’ll get talked out or just written in code and after the fact there’s not really a reference point to say, well, why did we do it this way? Why did we make this decision?
It also becomes difficult to communicate those sorts of things like across an organization, or this other team depends on this feature we’re building, or they have an interest in what’s getting built and they might not find out until after the fact that you decided to do it one way. And there’s way too much back and forth trying to have meetings with all these people, especially for remote companies to communicate these decisions synchronously, whether it be having one-on-one conversations or talking with people on a tool like Slack.
I think being more intentional about picking a documentation framework, a style and saying for each major feature or change that we’re going to work on, we’re going to take the approach to write these decisions down, why we’re doing something, if there’s any user research or anything backing it and here’s the path we’re taking and why.
Shane Hastie: The other part of that talk was the use of InnerSource. Tell us more about that.
InnerSource is using open source practices within an organization [07:56]
David Grizzanti: InnerSource is, I think, the definition that is floating around is the use of open source practices within an organization. And this is something I had become aware of maybe in 2018, 2019 when I was working at Comcast, and there’s an organization called InnerSource Commons that I mentioned in the talk that does a lot of work in this area, and the idea is to take the culture of openness and collaboration on software that happens in the open source space, but doing that for internal projects. And I think this is really beneficial for large organizations and small ones as well. But I found that, in my experience, at large organizations, it’s easy for teams to work in silos and not really know what’s going on across the company, and to rebuild similar tools or not be aware of all of the software that may exist at the company. And there’s benefits to reusing already built things even if they’re done internally.
I think the natural reaction is to say like, oh, is there an open source library for me to go do this work or I need a logging library or something. Somebody internally may have built that. It may just live in GitHub or your internal GitHub, someplace that you’re unaware of it.
So the idea of InnerSource is a few things. It’s creating this culture around all the software that’s built internally that only employees that the company can see, putting READMEs and contributing guides and release notes for your software, even if it’s only stuff that you’re using internally and seeing if there’s people at the company that are open to helping you build on the projects that you may be creating. A lot of developers now are interested in getting involved in open source and learning new languages, learning new tools. There may be opportunities even at your company to do that, to expand what you’re working.
And in that talk, I was riffing on the idea of organizational resilience with the InnerSource idea being that if you’re able to get outside contributors for projects that your team maintains within the company, then that may lessen the burden if people from your team leave, or if you have folks in the company using the tools you build, they could help improve the documentation by pointing out areas where it’s confusing for them on how to contribute or how to use the tool. I think oftentimes, the team is relatively small and stays the same for a long time. It’s easy to ignore documenting how the tool works or updating the README or updating the contributor guide because all that information becomes ingrained inside their heads.
Shane Hastie: Drawing on the ideas of the open source community sharing, spreading knowledge, building that organizational resilience. What is resilience?
Resilience is the ability to adapt to changes easily [10:28]
David Grizzanti: It’s a good question. I think it’s the ability to adapt to changes and those changes could be anything. It could be outages that your company’s facing for various reasons, like software going down, it’s 3:00 AM, making sure that it’s easy to fix something or bring something back up to running when it goes offline. It could be some kind of natural disaster, you lose a data center. Some of these practices maybe can help you resume operations quicker if you have good documentation or you have more people that understand the operations across the company. It could be loss of people through their own, they want to leave the company. The other thing I’ve seen affect companies pretty heavily is companies doing layoffs or reorgs. You shift all these people around and people gain or lose responsibility, but it’s really hard to shake old projects. So the better state things are in either through documentation or better guides, I think it can help companies adapt more easily.
Shane Hastie: Another thing that you have explored is the parallels between engineering and art. How is software engineering an art form?
Software engineering as an art form [11:29]
David Grizzanti: That idea was born from two things. I read this book sometime last year in the spring by Rick Rubin who’s a pretty famous music executive called The Creative Act. And I was talking to a friend of mine about this idea who read the book and had similar interest. He’s an engineer as well. And we were talking about how we both felt that it’s not always cut and dry our jobs of being an engineer. I think especially as you grow in the career ladder, your job gets a lot more nebulous of what your role is and how you should be handling stuff. And you’re responsible for crafting, whether it be architecture documents or design plans or navigating organizational dynamics. It’s a less of a exact science. So I went through some of the struggles that artists go through, whether it be writer’s block or having trouble coming up with ideas, like getting stuck in your head.
Rick talks about this idea of increasing the sensitivity of your antenna, this idea that it’s easy to get stuck in a spot and not really pay attention to the world around you. And I forget if he had this idea or I realized, this is the thing I was comparing it to is if you sit in the same room where you’re in your house all the time, you can easily ignore things because they become common to you, you look at them all the time, and then one day you’re sitting in a chair and you look up at the ceiling and you’re like, where did that spot or that hole come from? And you’re like, that’s probably been there for five years, but I never noticed it. So it’s easy to mistake or not see things that may be there.
So he goes through this idea of slowing down and paying more attention to what’s going on, and I was discussing how that could help engineers in their job of stepping back and paying more attention to maybe small things that are happening in the org or with your team, like their mood about projects or just picking up on social cues to see could that help you navigate some tricky situation at work with a project that one or two people may not be comfortable with and figuring out how to approach them or approach the situation and just seeing what a better path could be there.
I think I also talked about how to maybe help people work in their day to day if they’re feeling stuck. I think at the time I was thinking a lot about people being home and being in a singular space and what are some good ways to step back. And for me, at least, I know I need to have a creative space to get up and go for a walk or just explore and free my mind instead of just sitting at the desk all day. For people who are going to the office, that may be a little different. For me, I’m home most of the day working from home, so it helps to get up and free your mind a bit, eliminate distractions, that sort of thing.
I know that distractions are something that we deal with a lot anymore with people have their phone on their desk and things popping up all over their computer. The other thing for me is trying to focus very much on work at the time when I’m working, and then if I want to check my phone or something to step away and do that to keep the spaces separate.
Shane Hastie: What can artists tell us that the engineers don’t know?
Things engineers can learn from artists [14:21]
David Grizzanti: I mean I think artists… Not that engineers aren’t creative, but I think one of the things that maybe an artist thinks about with their work is they’re exploring new mediums and how can my art be more creative or explorative? And I think as engineers sometimes we either go one direction or the other or like, okay, I need to stay within my zone, that I understand this programming language, this architecture or whatever. And sometimes it’s good to just get into the habit of prototyping or building things, experiments or trying to craft something new to see where it takes you.
From talking with some other colleagues in the last few months, I think people tend to worry about spending time building something, a POC, whatever it is to see how it would turn out or if it would work. It’s like, oh, well that’s probably going to fail so I’m not even going to try it, or if I’m not going to take this all the way to be a successful feature or a successful product, I’m not going to actually build it. Let’s just talk it through and then if it’s not going to work, I won’t build it. I think, as an artist, draw or painter or whatever, and you’re constantly making new things, trying new ideas out, experimenting. And I think that’s important as engineers that we don’t lose that ability to just create and experiment on small things even if they go nowhere. Don’t be afraid to try something, even if it’s going to fail.
Shane Hastie: Don’t be afraid to try something, even if it’s going to fail.
David Grizzanti: Could be interesting even for ideas too. I think I’ve had something recently where I was trying to throw out an idea that I knew there was some pushback on and it did require building a small prototype, but even prior to building the prototype, it was like, let’s see how people feel about this idea.
And I got some feedback from a colleague that was like, appreciate that you do this or that you’ll put out ideas out there that may not even go anywhere, that may get a lot of pushback and will just die on the vine. And to me, that wasn’t super novel. I thought that was something that a lot of people were used to, but I think for them they were like, I think a lot of engineers will shy away. They think that their ideal will get shot down. If they get the sense that it may get rejected, they won’t really put the idea out there and they won’t put it out into the world. So I think that that’s something that is important to be open to feedback, but also who knows what might happen with this? It might be a great idea and you shy away from putting it out there because you get some feedback that’s negative in the beginning.
Shane Hastie: This kind allows me to segue into the conversation about developer productivity. What is productivity in the developer space?
Exploring developer productivity [16:47]
David Grizzanti: I think a lot of folks, and I think if you looked at productivity, the first thing that you would find if you search around Google is metrics to measure developer productivity. And I think the DORA framework that’s become or was popular for a long time and still is popular about having certain metrics that you can measure to see if an organization is successful is great, but it’s definitely not a one size fits all idea. I think it can be a little bit of a trap to assume that you can take some amount of metrics, one, two, three or four metrics and define the productivity of an engineering organization or of a single engineer.
I haven’t experienced this myself, but I know there’s been talk in the past of measuring lines of code for instance as a productivity metric, which is very dangerous, positive or negative lines, and I think we’ve moved away from that, but is number of deploys a day really a sign of quality software? I guess, if you’re deploying every day and it’s not failing, that’s good, but that doesn’t necessarily mean that your users are happy or you’re shipping quality features. It just means that you are deploying.
I think for me, what I’ve steered toward become more interested in is what does productivity mean for your team or your organization. If you’re building something that’s user facing, to me, you’re enabling engineers to be more productive. If they’re happy with their development environment, if they’re able to build the things that they want and need to and the users who are using their software that they’re building are happy and are getting the output that they want, versus just raw numbers of what might be happening with your software.
Shane Hastie: What are some of the traps that organizations fall into around this productivity conversation?
Productivity traps and mistakes [18:26]
David Grizzanti: Similar to the numbers thing I was mentioning, I do think it’s easy to try to adapt or pick a framework and assume that it’s going to fix challenges you have or give you a perfect insight into something that may be a cultural problem. I’ve seen this happen, not necessarily with productivity metrics but with other frameworks like OKRs for instance, which is a way of differently measuring goals. We use smart goals for a long time KPIs. I’ve seen organizations take those and like, we’re going to implement this tool OKRs and it will fix all of our problems. And it’s really just another way of establishing goals. It doesn’t necessarily make the company more productive or make your team more productive.
And I think that trap can happen with productivity as well. You can say, okay, the team’s only deploying once a month now we need to get our metrics down to deploy once a day, but that doesn’t mean that the software is, again, better quality or it’s actually delivering value. The engineers could say, okay, my target is to deploy X times this week or this month now, so let’s build towards that, versus measuring real value.
I know that there’s been a lot of research and interest in doing developer surveys, like measuring how happy developers are using that as a metric instead of these more quantitative things. And I think that that has some promise, though I do feel like developers and everyone gets survey fatigue. People don’t want to be constantly surveyed. So this interesting balance of figuring out where the right tone is to strike with that stuff. Don’t over ask. Don’t ask too many questions too often. Find out when a good place to get this feedback is.
Actually, at the QCon conference as I found it was interesting and I think this is the common thing of asking people for feedback at the moment that the thing is occurring. So at the end of the talks, there’s little notes that you can pick up that are green, yellow, and red to rate the presentation. And I think I’ve been thinking about how could you incorporate that style of feedback with interactions with users who may be using your software to measure their happiness with the product or with the feature instead of having to ask them after the fact, sort of like the traditional NPS style user feedback.
Shane Hastie: Getting feedback early and often.
David Grizzanti: Yes.
Shane Hastie: What does this mean for engineering leaders?
Advice for engineering leaders [20:36]
David Grizzanti: I think that they need to be introspective about what they’re looking for out of these metrics and not just adopting a framework because it might be the new thing that they’re reading about online, and really trying to figure out what value they’re trying to get out of the data and what problems it’s solving. I think oftentimes, we adopt the framework where there’s not really a core problem being solved or the framework’s, it’s just going to introduce another set of data that doesn’t necessarily solve the specific challenge that we’re having.
I think we oftentimes say, well, having the data is beneficial, that way if we have a problem where we want to see what’s going on, we’ll have all this information. But it’s not that knowing how many times a deploy happens a day or something is solving a specific challenge the company may have. So I think just really getting to the crux of what the company’s challenges are or what the organization’s challenges are, and thinking about do any of these productivity metrics or frameworks help solve your challenges, and is there more of a culture problem that needs to be addressed and not something that a framework can necessarily solve?
Shane Hastie: And extending the leadership conversation, what advice would you have for individual contributors who are considering or are stepping up into leadership roles? What does a new leader in a technical environment need to know?
David Grizzanti: I think for me, one of the things that I’ve found, it’s helped me in that journey, an advice that I would give people starting out in that similar spot, whether it’s you’re at a company for a long time and you’re looking to take on that role or you’re starting at a new company in this engineering leader, individual contributor track is really try to get to know, not only the people that are in the organization that you’re joining or you’re a part of, but also the technology challenges and also the more social constructs within that organization or company, and be as approachable as possible.
I think I’ve found that starting a new job after being at the same company for a long time, I definitely approached it very open-minded and spent probably the first 30 to 60 days just meeting with as many people as I could, not worrying too much about what the expectations of I’m coming in at this role or I’m becoming a leader, I need to hit these targets of contributing exactly this much code or fixing these problems.
Spend the time understanding the challenges, giving advice based on your experience where you can, don’t be afraid to bring up new ideas with people, and show people that you’re willing to listen and adapt to what the organization needs, instead of bringing in preconceived notions about this worked at my last company or in an earlier role, I think we should tackle it this way.
It’s definitely challenging to go from a very comfortable environment where you know your role and you’re operating a little bit on autopilot to something that’s totally different or you have to learn a lot within the first month or two. But I think for, even though that’s a little scary, I think it’s nice to be challenged and step outside your comfort zone and just learn a lot of new stuff and see where you can contribute, but also still be taking on new challenges and learning.
Shane Hastie: Take on new challenges and learn.
David Grizzanti: Always be learning.
Shane Hastie: Always be learning. Thank you.
David, a lot of really interesting and useful stuff here. If people want to continue the conversation, where do they find you?
David Grizzanti: I am still on X, formerly Twitter at dgrizzanti, my last name. You can also reach out to me on LinkedIn. I have a profile on InfoQ as well where I’ve written a few articles.
Shane Hastie: You have indeed.
So David, thank you very much for taking the time to talk to us today. Great to have you on the podcast.
David Grizzanti: Thanks so much for having me.
Mentioned
.
From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.
Teacher Retirement System of Texas increased its stake in MongoDB, Inc. (
