Article originally posted on InfoQ. Visit InfoQ

2022 was another year of significant technological innovations and trends in the software industry and communities. The InfoQ podcast co-hosts met last month to discuss the major trends from 2022 and what to watch in 2023. InfoQ News Manager Daniel Bryant led and moderated the discussion. The panelists include Wesley Reisz, Shane Hastie, Thomas Betts, and Srini Penchikala.

This article is a summary of the 2022 software trends podcast. Check out the podcast episode and the full transcript of the discussion.

The main topics and themes discussed in the podcast were as follows:

• Architecture Decision Records (ADRs)
• Role of Staff Plus engineers
• Platform as a product
• AI-based assistants like ChatGPT
• Sustainability and Green IT
• Culture Design

The sections below summarize the key trends discussed by the group.

The Shift in Architect’s Role

One of the key personas at InfoQ is the architecture persona, the role of the architect, and how they can serve the teams around them and contribute to the success of software development projects and initiatives in their organizations. The architect’s role is shifting to be an active part of the development lifecycle, assisting the application teams with architecture decisions in a collaborative way rather than enforcing, and communicating decisions to the stakeholders at different levels of the organization. 

The pandemic and hybrid work situations influenced the teams to communicate more asynchronously. The architect’s role in this context has become even more critical in terms of writing things down and ensuring the decisions are in the organization’s best interest for the long term. Diagrams of the architecture solutions are not sufficient anymore. They need to justify why a particular solution was designed that way in the first place.

Architecture decision records (ADRs) have been around for a while but are now being adopted by more companies. ADRs help in several ways, specifically in figuring out the architectural choices and writing them down. The teams can then discuss why they made a specific decision. You see the pros and the cons, and then it’s more of a collaborative process.

ADRs can be used to document and version the architecture decisions with details like “What is the decision you’re trying to make? What are the possible options you’re considering? What are the pros and cons of each option?” All of this information can be checked into a centralized Git repo. This way, the teams can establish the shared context of why they chose a specific technology or software over the alternate options.

Architecture decision-making should be collaborative, and ADRs can play a significant role in the process. Architects can use them first, establish the pattern, and figure out what works within their company. It’s also living documentation. Teams can change their mind and update the diagrams that are out of date. ADRs put it in a simple text format. They can update it, and then if they want to create a diagram, Mermaid diagrams are an excellent choice. Maintaining this type of living documentation, and the culture that supports it is also important.

People working as architects should avoid the “ivory tower architect” approach and work actively to embed the architecture outcomes and deliverables with the project in the same Git repo.

Increased Visibility of the “Staff Plus” Role

With the increasing adoption of the “Staff Plus” role in IT organizations, the role of each individual is changing positively. At a certain point in our careers, we as architects used to think, “What am I going to do now? I’ve gotten to this point where I am an architect. I’ve gotten to be a manager and into a director role.” Now, with Staff Plus, senior leaders at companies are starting to think, “What is the roadmap beyond that staff level?” The companies intentionally looking at Staff Plus is a significant development in this area.

Architecture is also getting the proper focus as a discipline and a software craftsmanship rather than just diagrams and artifacts. That’s where the architects are becoming more valuable to the enterprise because of what they can do and how they can contribute to the teams, rather than just creating some PowerPoint slide decks and throwing them over the wall. Most architects are becoming hands-on and involved throughout the software development life cycle. The architecture process is now iterative and incremental in delivering value to the customers.

Having the Staff Plus engineer position in the organization tends to be a T-shaped role where the individual can be broad across many disciplines but have deep expertise in a particular area. This role can help with thinking about the cross-cutting concerns, the big projects, and the big picture of how it solves multiple problems.

Charity Majors gave a talk at QCon San Francisco titled “The Engineer/Manager Pendulum” on consciously and deliberately bouncing in and out of that senior Staff Plus/Architect role and into management and then bouncing back. This way, people can build many deep competencies in different spaces and move back and forth. Managing your career like a product and deliberately making choices to help grow your career.

One of the trends we have seen since the beginning of the pandemic is the great resignation. 30% of people changed jobs, and 70%, according to some studies, are actively dissatisfied with their current position. This prompted a leadership level to create opportunities for people to move as their interests shifted without losing the people to other companies.

In some organizations, the senior IC or Staff Plus positions are being created more as evolved opportunities rather than appointed job postings. So, it’s not like the senior leadership is saying, “Okay, we will make you a senior Staff Plus engineer.” It’s the other way around. The team members can contribute not only technically but also organizationally. They’re able to manage their own people and the stakeholders and make the Staff Plus opportunities happen.

Platform Engineering and Reducing Developer Cognitive Load

When DevOps practices came out a few years ago, it was meant to be the best of both Dev and Ops worlds. It helped with many things, but in the process, we took cognitive load on our teams. It got really high, and burnout became an issue. Trying to keep a mental model of all the things a team has to deal with today, from Kubernetes to Istio to the sidecars to ingress—and then you’ve got to write code—is getting quite a bit of cognitive overload. 

Platform engineering is another area that’s getting a lot of attention lately. The platform is a massive lever, and the idea with platform teams is how we can pull that lever and start to reduce the cognitive load. If you get it right, you can enable all the stream-aligned teams (Team Topologies) to deliver value.

To reduce the friction in development teams, the platform teams can help create internal developer platforms and offer self-service capabilities to onboard and host their apps. These platforms should support tasks like automatically provisioning infrastructure, seamlessly pushing app code in a CI/CD pipeline, and verifying the quality of the infrastructure components before giving the teams the go-ahead to start deploying their apps to the platform.

Internal developer portals like Spotify’s Backstage, a CNCF open source project, Netflix’s paved road, and other similar tools can help with goals of platform engineering.

We think next year we’ll see a lot more focus on internal development platforms.

Data Mesh and Data Platform as a Product

Data mesh is an important architecture model to help manage  data from different sources, which are consumed by different clients and consumers, with a governance layer built-in.

Companies struggle with implementing a data mesh because they have to create a platform that allows them to take charge of the principles and tackle various individual data products. The goal is to have a standardized mesh where everyone can put the data in and get the data out they need, as opposed to having the bottlenecks.

We’re also seeing more focus on platforms as products. A new and interesting trend in this area is to have a product manager on a platform. It’s an interesting role because you have to be empathic and be able to engage with the developers, your customers, and the users. You have to be good at stakeholder management because the senior leadership team often asks questions like, “Why am I paying for this platform? What value is it adding?” The platform product manager needs to articulate that the platform engineering is an enabler and that we’re investing in solid foundations, either for platforms in terms of applications or platforms in terms of data.

IT is not just a cost center anymore; software is what’s enabling your business to be more productive. And platform engineering has a vital role in software development and maintenance.

Emerging Trends in AI, ML, and Data Engineering

Data, similar to architecture and security, is going through the “shift left” approach. Data is no longer just something you store somewhere. It is becoming a first-class citizen in terms of modeling, transformation, and processing. The end-to-end automated data pipelines are getting more attention because you cannot have the data in silos or duplication of data and the lower quality of the data—all those problems. +The database is one of the solutions for storing the data, but other trends like streaming first architectures where the data is coming in terms of data streams. How we process that kind of data is what’s driving the emerging architectures in the data engineering space.

There are also other trends, like streaming warehouses which focus on how we capture and analyze the data coming in terms of streams. Looking at all the major developments in this area, there are data-related trends, data management, and data engineering. Machine learning and artificial intelligence are the second area. The infrastructure to make all of this happen, data platforms and everything, is the third area currently going through a lot of innovation.

We need to think about data upfront as part of the entire system. How do we make sure we have observability, not just of the system, but of the data to make sure that the data is flowing through properly? Are we going to use AI models? Can we get our data in a way that we can feed it into a machine-learning model so we can get some of those benefits? All of these things  have to be considered. So, that’s where architecture has to start thinking a little differently, not just here’s the product. Creating data separately as a focus on the data and architect for the data, it’s a different way of thinking.

Data, as they say, is the second most important asset of any company after the people. So yes, we definitely need to give it as much attention as possible. Data is going through a similar evolution that code and architecture have gone through in the past. There is a continuous CI/CD approach for data as well in terms of receiving the data, ingesting it, processing it, and versioning the data.

In terms of machine learning, probably there’s no other technology area that has gone through the same level of innovation as machine learning and AI did in 2022. GitHub’s Copilot project is one of the interesting innovations. It has been talked about as a tool to improve developers’ productivity. We’ve heard from some developers that Copilot has made them 100% more productive, so pretty much 2x productivity. The developers say they don’t write any basic functions anymore. They don’t need to remember how they’re written. Just ask Copilot and it generates all the code for them. They don’t even use websites like StackOverflow anymore. We also are seeing new technologies like ChatGPT that are getting a lot of attention in terms of how they can change not only developers’ lives but everybody else’s lives.

Legality, Ethics, and Sustainability Challenges with Computing

The software development community needs to be responsible not only in the legal area but also in ethical and sustainability domains. We need to focus on ethical aspects of ensuring that the products we build are good for the environment (green computing) and the broader community (social computing).

We should remember that inherent biases are built into any of the models. The data that these models are sampled and read off is what the models have built in. With new tools like ChatGPT, interacting with AI is no longer limited to AI/ML subject matter experts; pretty much anyone can use these AI assistant tools to generate new programs and scripts. This is why it’s more critical now than ever to be consciously and intentionally responsible for the products we deliver.

Everyone involved in the software industry in any capacity should start asking questions like “Can we make this a green solution?” “Where am I running this? Is it using green energy or is it running in one of the data centers that’s all coal-powered energy?”

Adrian Cockcroft recently spoke on DevSusOps topics about bringing sustainability concerns to development and operations. The idea is to manage sustainability as a non-functional requirement for developing and operating software systems. This includes measuring and reducing the carbon footprint, water stewardship, social responsibility in supply chains, and recycling to create a circular economy. You can use tools like the Cloud Carbon Footprint from Thoughtworks to estimate the carbon footprint of your apps hosted on the cloud providers.

What Will the Future of Work Look Like?

Another area that’s going to go through a major transformation in 2023 is the hybrid work model. Hybrid work environments, if done right, can be the best of both worlds: WFH (Work from Home) and RTO (Return to Office). But if it’s not managed properly, hybrid hell can become real. The reality of people commuting to the office, just because it’s a mandate, two or three days a week, is not coordinated well. People come into the office and spend most of the day on Zoom calls. We’ve got to start being deliberate about why we bring people together. There is a huge value in coming together in person.

Suppose the senior leadership teams are going to bring people together. In that case, they need to consider the cost for the organization, the environment, and the people and make sure that the benefit outweighs the cost. If we’re coming in, maybe it is one day a week, it should be the same day for everybody on the team and all the stakeholders we need to work with. This way, we can have collaborative conversations in person and leverage the humanity of it.

The in-person mandate cannot be “just come to the office Tuesday, Wednesday, and Thursday every week.” It has to be based on context and product lifecycle, not just driven by the calendar. If we are doing sprint planning, retrospectives, or brainstorming, that requires everybody, preferably in person so that they can collaborate for one or two days. Once the development phase starts, not everybody has to be in the office.

Companies should have a deliberate reason to bring people into offices. They should also let go of how we measure. It’s not about hours in front of a screen. It’s about outcomes. 

If we think of the impact from a green and climate perspective, some organizations and  governments are shifting to four-day work weeks. The studies are amazing. In every organization that has shifted to the four-day work week, productivity has stayed the same or gone up. People are more focused because now they’ve only got four days to get the work done.

Getting it right, bringing people together, and connecting as human beings allows you to be more comfortable and safer in your environment. It will enable you to be present better. The bond the in-person collaborations can cultivate is significant.

Looking Ahead to 2023

What do we as a group want to see in 2023? A deliberate culture design in organizations and some of the experiments; the four-day work week; more and more organizations bringing that on; outcome focused, humanistic workplaces.

We think we’re right at the inflection point with artificial intelligence becoming mainstream. An AI assistant can enhance the role of an architect. We can just ask the chatbot for helpful information, it can respond accordingly, help us think through, and give us the ability to be 10x better in delivering value to our stakeholders. We also think we’re going to see something come about in AI in the next 12 months that we didn’t expect to see, that it just becomes very mainstream, and we all start using it.

ChatGPT and other future AI solutions can do a better job helping people, but we don’t think they will replace humans anytime soon. We look forward to how data and AI/ML technologies can help with all aspects of our lives at the individual level, the community level, and the national and government level. How can AI help at all levels, not only in our offices, at work, and in our personal lives, but also in other areas like healthcare, governments, etc.?

The deeper we get into technical issues, the more we find out it’s about people, about organizations, and about communication. The technical stuff that comes along is not the hard part. We want to see the continuation of platform conversation next year, building stronger teams, being able to do more with less, and reducing cognitive load so that people can develop software and be happy and healthy doing it.

Other trends like low-code, no-code solutions are going to enable citizen developers. The way the world is communicating and collaborating is also changing, in most cases leveraging virtual reality (VR) based technologies.

References mentioned in the podcast: